Sophia Spitulnik

BrainGate Data Visualization

An interactive data visualization for a 
neuroscience research group.

Data Visulization Preview

Background

Overview: BrainGate, is a technology that allows patients to control computer interfaces with their brains and is developed at Brown University. Their goal is to restore the independence of people affected by various neurologic diseases. While patients use this technology, the BrainGate researchers collect data on the patients’ cursor movements and clicks. In order to improve their technology, the researchers needed an intuitive way to visualize and analyze the data they collect. I worked with a team of Computer Science students at Brown University to design and develop an interactive data visualization tool for the BrainGate researchers. This project was done as part of a Computer Science class at Brown titled “Creating Modern Web Applications.”

Scope: During this project we: Interviewed our users (the BrainGate researchers), wrote user stories, wireframed different design solutions, created high fidelity mockups, wrote specifications and requirements, researched frontend and backend technologies, developed a working prototype, presented to our class and our client, had weekly meetings with the BrainGate engineering team to get feedback on ideas, wireframes, mockups and prototypes.

Tools: Github, Sketch, Principle, Illustrator, Google drive, pens and paper, whiteboards, Sublime

Team:

Jean

Jean: Front End Developer

Marcin

Marcin: Back End Developer

Zhengyi

Zhengyi: Front End Developer

Sophia

Sophia (me!): Experience Designer

Original Problem Statement

Our client, the BrainGate engineering team, needed a way for the BrainGate researchers to better visualize their data.

Users

The BrainGate Researchers

Interviews: We interviewed our users to learn about their jobs, the tools they use and their day to day experiences. This project is very unique in that we have a very small and specific user group that could quickly give us feedback and who can continue to improve their program as they use it.

researchers

A summary of what we learned: The BrainGate researchers observe the patients and collect a lot of numerical data. For example, they collect data on the movement of the cursor over time and how accurate the participant was with the cursor. They take qualitative notes on their sessions with the patients in Word documents. The researchers want to be able to see this data in a visual format, instead of trying to compare or analyze numbers. The researchers are interested in troubleshooting the BrainGate system by comparing the results of different settings, and visualizing the data collected after each session. Specifically, they are interested in visualizing the position and trajectory of the cursor over time, and being able to create filters for the data collected.

New Problem Statement

After conducting user interviews and learning more about the problem we redefined our problem statement:

The BrainGate researchers need a way to organize their data, visualize patient cursor movements overtime and annotate data based on their observations.

Before this project, the BrainGate researchers had no real way to visualize the data they collect about the patients’ cursor movement overtime other than just data coordinates, time stamps and what as provided to them in matlab. They also collect notes about their sessions with the patients using this technology in Word documents and record things like “at 0:45 seconds someone made a loud noise in the background and caused a distraction.” The notes were disconnected from the data points. All of their data was stored in MatLab files and not in a database.

Ideation and Wireframing

Each time I met with the BrainGate team I got some feedback on the wireframes and continued to iterate. See the mockups section for descriptions of design decisions and the functions of specific parts of this data visualization.

sketch
sketch sketch sketch sketch wireframe wireframe wireframe wireframe wireframe wireframe wireframe wireframe wireframe wireframe wireframe wireframe wireframe wireframe

Mockups

A user selects the set of data that they would like to view through filters. In this GIF, the user selects the date that the data was collected on.

timeline preview

Other filters that the user can use to select a set of data are the patient's name, tags, parameters and start and end time of the data.

timeline preview

Once the data set is selected, the researcher can view the data in the graph section.

In these mockups, the user is viewing a set of data where a patient is playing a "game" to practice using the system. The patient is trying to move their cursor to the light blue targets. In the future, any cursor data (not just game data) could be analyzed using this program.

timeline preview

The red dot is the “current” X and Y location of the patient’s cursor. A dark green line shows the future of the cursor's pathway and a light green line shows the past of the cursor's pathway.

timeline preview

There is also a “full view” where the researchers can view all of the cursor movements for the entire session with the patient.

timeline preview

The user can annotate the data by adding notes at specific times. For example, if a researcher sees a cursor jump at 0:20 seconds, they can add a note at that time with the tag “jump”. A indication of this note will appear on the timeline.

add note

If a user hovers over one of the notes they will see a preview on the timeline at the time that the note was taken. If a user clicks on one of the notes it takes them to the time the note was taken on the timeline and graph.

timeline

timeline

If the user hovers over a timeline marker then the note is lightly highlighted and if the user clicks on the timeline marker then the note is highlighted. They can also see a preview of the graph at that time by hovering.

timeline

timeline

One of the biggest challenges was visualizing X and Y coordinates over time. I used inspiration from YouTube as well as other video and music players to design a timeline that the researchers could use to play through or scrub through the “recording” of each session. The timeline includes markers to show that a note was added at a specific time.

timeline

Prototype

We built a working prototype of this program. It was a great learning experience to work with a team of engineers to actually build something that I designed. We used a local MongoDB database (with our data converted to JSON from the Matlab files), svg.js library for the graph and timeline and RESTful API to load the data into the program.

Read more about the design of the program here.

If you are interested in looking at our documentation for this project, send me a message.

Reflection

The clients and researchers gave very positive feedback to our final presentation of our prototype. It was very rewarding to see that something I worked on would be used to help hardworking researchers make a positive impact in patients’ lives. It was a great learning experience to work with a client, a group of users as well as a team of engineers. I learned about understanding users’ needs, asking good questions, designing a creative experience, communicating my design decisions to clients and engineers and web app technologies. This project will be a memorable one because of the awesome people that I had the opportunity to work with and because the project will help in BrainGate’s mission to give patients with neurological disorders more independence.