Brain power

Pretend you have just returned to your office after an impromptu meeting. “Now, where was I?” you think. You look down at your desk for clues. Papers are scattered; files overlap. You look up at your computer monitor for a hint. But the screensaver, featuring your two pet dogs, stares back at you predictably. You touch the space bar, and instantly your eyes meet the dizzying grid of a spreadsheet. You gaze through the window at an anonymous high-rise across the street, searching for something to jog your memory and get you back on track. “Now, where was I?” you continue to think.

Psychology professor Dennis Proffitt believes he can have you answering that question much sooner, thanks to a computer system interface called the InfoCockpit, which is up and running in a new Cognitive Science Laboratory in Gilmer Hall. Proffitt, who directs the lab and oversees the Arts & Sciences cognitive science undergraduate degree program, said U.Va. students and researchers now are able to investigate applied technologies, like the InfoCockpit, that build upon science’s improved understanding of how the brain works and how human cognition, perception and memory relate to each other.

According to Proffitt, the conventional office environment is not effective for information retention because it lacks the contextual cues that we rely on to remember something. “If I ask you what you had for lunch last Friday, recalling where you were will help you figure it out,” he said. “Distinctive places and people trigger memories, and that’s what we want your computer system to do.”

The InfoCockpit provides location and place cues to enhance recall. Information is distributed across three computer monitors, allowing users to associate the information with a specific location. Projected on a curved screen behind the monitors is a panorama that slowly, almost imperceptibly, rotates 360 degrees. Combined with a surround-sound system, it creates an ambient sense of place.

Proffitt envisions a work environment in which users work on project x in San Francisco, project y at the lake and project z on the Lawn, for example. “So, all the unfinished business you have with a specific project could be triggered by returning to the place you always go to when you work on that project,” said Proffitt, who has a smaller version of the Info-Cockpit in his office.

“Neither the images or sound alone do much to trigger memory, but together they work incredibly well to augment memory,” he said. The lab’s controlled studies show that users remember as much as 130 percent more than those in the conventional media setups. Of course, to be effective, the environment must be engaging, but not so engaging that it causes distraction. “That’s the art to it,” said Proffitt.

He is beginning to look at how InfoCockpit technology might also be effective at changing mood and consequently increasing productivity.

Lab staff member Jason Cervenka (Cognitive Science ’03) has developed a clinical application for the technology. Soon the panoramas of the InfoCockpit will be projected in several inpatient rooms at U.Va.’s medical center. “Patients who have windows have better recovery rates than patients without windows,” said Proffitt. “We think the panoramas will be even better. If just one patient a week stays one less day in the hospital, the [cost savings] would pay for the entire system setup.”

The InfoCockpit is an interdisciplinary initiative, drawing from the knowledge of not only cognitive psychologists but also engineers, computer scientists, neuroscientists, graphic artists, designers and others. “Anything we do to improve our understanding of how to use the technology in one context impacts the other,” Proffitt said. The lab’s interdisciplinary nature ensures the creation of technology that is both useful and fun, he added.

“What we’re all about is making technology accommodate people as opposed to people having to accommodate technology,” said Proffitt.

The lab is investigating ways to use brain imaging to improve the people-technology relationship, looking, for example, at how a computer might adjust to the way our brains actually work. The brain can function concurrently in some ways, but not others; you can drive and talk at the same time, but you can’t carry on a conversation while doing mental arithmetic. “Technology has no respect for this,” said Proffitt. “Computers interrupt people all the time, demanding that they do a task when their brains are already thinking about something else.”

To remedy this, the lab and collaborators developed an automated task scheduler that detects real-time brain activity using functional near-infrared imaging (fNIR) and adjusts information flow to accommodate the user’s available cognitive resources. The fNIR system (called the Optical Tomographic Imaging Spectrometer, or OTIS) directs low-level infrared light, which is less intense than sunlight, at Broca’s area, the region of the brain that controls language processing. Sensors in the headpiece then pick up reflected light, which indicates the level of neural activity.

Initial OTIS research sought to improve human performance by controlling the flow of nonessential information. But Peter Wubbels (Cognitive Science ’05) and Margaret Perschy (Computer Engineering ’05) are using the OTIS system to develop a communication interface for people with locked-in syndrome, a rare brain disorder that completely paralyzes individuals, leaving them conscious and aware but unable to communicate.

The prototype system allows the user to spell words by “selecting” a desired letter by counting silently. The resulting neural activation in Broca’s area is then picked up by the OTIS system. “If thinking in language is something a person can control, it can be used as a switch. By counting or not counting, a person can answer yes-or-no questions, move a cursor, use a speller and so forth,” Proffitt explained.

Wubbels and Perschy met with locked-in patients and their families in Atlanta to learn what communication systems would be most beneficial. Some locked-in patients most want to communicate basic human needs, like feelings of thirst, hunger, discomfort or pain, while others desire a means of sophisticated interaction. “It’s so rare, especially for undergraduates, to see how their work has any bearing on the rest of society,” said Wubbels, “so to meet firsthand the people who have a real need for your work is incredibly motivating.”

“Ultimately, the lab is an educational resource for students,” said Proffitt. In addition to several graduate students, postdocs and staff members working in the lab, 10 undergraduates are getting course credit by working as research assistants. “I think one function of a major research university is providing undergraduate students with an opportunity to not only be taught by people doing the research, but also to contribute to research.”