Mind and Body Join Robotics in New Research Project

Three UC Merced researchers are connecting eye-tracking hardware with a robotic exoskeletal arm to further understand the mind-body connection and, someday, to help people with disabilities have better use of their limbs.

A National Science Foundation Major Research Instrumentation (MRI) grant and UC Merced’s Office of Research and Economic Development are providing $250,000 for the mechanical arm so professors Ramesh Balasubramaniam, Michael Spivey and Stefano Carpin can conduct interdisciplinary research into the cognitive and robotic aspects of motor control and human-machine interfaces.

“Research indicates that cognition is a continuous thing,” said Spivey, a cognitive science professor with the School of Social Sciences, Humanities and Arts. “While the mind is busy telling the body how to act, the body is also telling the mind how to think.”

The three-year project is a prime example of UC Merced’s interdisciplinary spirit and research the campus is rapidly becoming internationally renowned for: robotics and embodied cognition, or the theory that each person is more than just his or her brain.

Embodied cognition research shows that aspects of the body, including the motor system, the perceptual system, the body's interactions with the environment and our own assumptions about the world, are built into both the body and the brain. Researchers believe those physical aspects influence high-level mental constructs such as concepts and categories, and human performance on cognitive tasks, such as reasoning or judgment.

The eye-tracking headset in Spivey’s lab has already indicated eye movements can influence and even “nudge” people’s moral decisions.

“We’re finding out that the mind depends on movements,” Spivey said. “For example, people who have movement disorders actually think a little differently about their world because they do not have the same sensorimotor experiences as other people.”

By integrating an autonomous eye-tracking headset with the robotic arm, the researchers will get a better understanding of the dynamics of reaching and grasping movements at the millisecond level.

Balasubramaniam, a cognitive scientist, plans to use the transcranial magnetic brain stimulation (TMS) in his lab to see how interrupting or changing what the arm does while someone is wearing the exoskeleton can help the researchers understand how the mind works with and on movement. He studies motor control with an eye toward rehabilitation for people with physical and cognitive disabilities.

“We know the brain takes advantage of the natural arm dynamics, but there are many experiments we can do to help us understand the parameters the brain uses to plan movements,” he said. “By using the eye-tracker, the arm and TMS, we can begin to parse the cortical and peripheral contributions to motor planning.”

The project presents Carpin, an electrical engineering and computer science professor with the School of Engineering, with a particular set of challenges.

“As more robots are operated in close proximity to, or paired with, people, there’s an increased focus on high-level movements,” Carpin said. “Because the robotic arm will be coupled with people, there is a tricky set of engineering and planning questions, and I’ll have to work on millisecond time scales in designing robotic solutions.”

The project also makes UC Merced one of the first universities in the country to have all these embodied cognition resources in one place, including such a large group of researchers with expertise in the field.

“This project represents a true collaboration across disciplines at UC Merced, bringing together 12 researchers from cognitive and information sciences, electrical engineering and computer science, mechanical engineering and other groups,” said Balasubramaniam, the principal investigator on the $250,000 grant. “The project will involve graduate and undergraduate students, too, offering them research experience and education they will not get elsewhere.”

Part of the project will involve workshops to train science, technology, engineering and math students from the UC and other schools in embodied cognition, though schedules have not been detailed yet.

The relationship between cognitive science and engineering is steadily expanding each year, Carpin said, in part because UC Merced is still a small campus where researchers and students all work closely together.

“We’ve got a really unique research facility, which one of the NSF reviewers pointed out in our grant acceptance,” Spivey said. “Other universities don’t have this because they aren’t as interdisciplinary as we are — it’s in our DNA here.”