BrainGate Data Visualization
An interactive data visualization for a neuroscience research group.
Overview: BrainGate, is a technology that allows patients to control computer interfaces with their brains and is developed at Brown University. Their goal is to restore the independence of people affected by various neurologic diseases. While patients use this technology, the BrainGate researchers collect data on the patients’ cursor movements and clicks. In order to improve their technology, the researchers needed an intuitive way to visualize and analyze the data they collect. I worked with a team of Computer Science students at Brown University to design and develop an interactive data visualization tool for the BrainGate researchers. This project was done as part of a Computer Science class at Brown titled “Creating Modern Web Applications.”
Scope: During this project we: Interviewed our users (the BrainGate researchers), wrote user stories, wireframed different design solutions, created high fidelity mockups, wrote specifications and requirements, researched frontend and backend technologies, developed a working prototype, presented to our class and our client, had weekly meetings with the BrainGate engineering team to get feedback on ideas, wireframes, mockups and prototypes.
Tools: Github, Sketch, Principle, Illustrator, Google drive, pens and paper, whiteboards, Sublime
Original Problem Statement
Our client, the BrainGate engineering team, needed a way for the BrainGate researchers to better visualize their data.
The BrainGate Researchers
Interviews: We interviewed our users to learn about their jobs, the tools they use and their day to day experiences. This project is very unique in that we have a very small and specific user group that could quickly give us feedback and who can continue to improve their program as they use it.
A summary of what we learned: The BrainGate researchers observe the patients and collect a lot of numerical data. For example, they collect data on the movement of the cursor over time and how accurate the participant was with the cursor. They take qualitative notes on their sessions with the patients in Word documents. The researchers want to be able to see this data in a visual format, instead of trying to compare or analyze numbers. The researchers are interested in troubleshooting the BrainGate system by comparing the results of different settings, and visualizing the data collected after each session. Specifically, they are interested in visualizing the position and trajectory of the cursor over time, and being able to create filters for the data collected.
New Problem Statement
After conducting user interviews and learning more about the problem we redefined our problem statement:
The BrainGate researchers need a way to organize their data, visualize patient cursor movements overtime and annotate data based on their observations.
Before this project, the BrainGate researchers had no real way to visualize the data they collect about the patients’ cursor movement overtime other than just data coordinates, time stamps and what as provided to them in matlab. They also collect notes about their sessions with the patients using this technology in Word documents and record things like “at 0:45 seconds someone made a loud noise in the background and caused a distraction.” The notes were disconnected from the data points. All of their data was stored in MatLab files and not in a database.
Ideation and Wireframing
Each time I met with the BrainGate team I got some feedback on the wireframes and continued to iterate. See the mockups section for descriptions of design decisions and the functions of specific parts of this data visualization.
A user selects the set of data that they would like to view through filters. In this GIF, the user selects the date that the data was collected on.
Other filters that the user can use to select a set of data are the patient's name, tags, parameters and start and end time of the data.
Once the data set is selected, the researcher can view the data in the graph section.
In these mockups, the user is viewing a set of data where a patient is playing a "game" to practice using the system. The patient is trying to move their cursor to the light blue targets. In the future, any cursor data (not just game data) could be analyzed using this program.
The red dot is the “current” X and Y location of the patient’s cursor. A dark green line shows the future of the cursor's pathway and a light green line shows the past of the cursor's pathway.
There is also a “full view” where the researchers can view all of the cursor movements for the entire session with the patient.
The user can annotate the data by adding notes at specific times. For example, if a researcher sees a cursor jump at 0:20 seconds, they can add a note at that time with the tag “jump”. A indication of this note will appear on the timeline.
If a user hovers over one of the notes they will see a preview on the timeline at the time that the note was taken. If a user clicks on one of the notes it takes them to the time the note was taken on the timeline and graph.
If the user hovers over a timeline marker then the note is lightly highlighted and if the user clicks on the timeline marker then the note is highlighted. They can also see a preview of the graph at that time by hovering.
One of the biggest challenges was visualizing X and Y coordinates over time. I used inspiration from YouTube as well as other video and music players to design a timeline that the researchers could use to play through or scrub through the “recording” of each session. The timeline includes markers to show that a note was added at a specific time.
We built a working prototype of this program. It was a great learning experience to work with a team of engineers to actually build something that I designed. We used a local MongoDB database (with our data converted to JSON from the Matlab files), svg.js library for the graph and timeline and RESTful API to load the data into the program.
If you are interested in looking at our documentation for this project, send me a message.
The clients and researchers gave very positive feedback to our final presentation of our prototype. It was very rewarding to see that something I worked on would be used to help hardworking researchers make a positive impact in patients’ lives. It was a great learning experience to work with a client, a group of users as well as a team of engineers. I learned about understanding users’ needs, asking good questions, designing a creative experience, communicating my design decisions to clients and engineers and web app technologies. This project will be a memorable one because of the awesome people that I had the opportunity to work with and because the project will help in BrainGate’s mission to give patients with neurological disorders more independence.