Partnering with the CS department has been really productive over the last semester. We just submitted another new paper to the International Symposium on Robotics Research (ISRR). I am truly grateful for being able to work with such a talented and generous group of faculty and students.

Communicating Robot Arm Motion Intent Through Mixed Reality Head-mounted Displays

Draft Abstract:

Abstract Efficient motion intent communication is necessary for safe and collaborative
work environments with collocated humans and robots. Humans efficiently
communicate their motion intent to other humans through gestures, gaze, and social
cues. However, robots often have difficulty efficiently communicating their motion
intent to humans via these methods. Many existing methods for robot motion intent
communication rely on 2D displays, which require the human to continually pause
their work and check a visualization. We propose a mixed reality head-mounted
display visualization of the proposed robot motion over the wearer’s real-world view
of the robot and its environment. To evaluate the effectiveness of this system against
a 2D display visualization and against no visualization, we asked 32 participants to
labeled different robot arm motions as either colliding or non-colliding with blocks
on a table. We found a 16% increase in accuracy with a 62% decrease in the time it
took to complete the task compared to the next best system. This demonstrates that a
mixed-reality HMD allows a human to more quickly and accurately tell where the
robot is going to move than the compared baselines.

Rosen, E., Whitney, D., Phillips E., Chen, G., Tompkin, J., Konidaris, G., & Tellex S. (Submitted). Communicating robot arm motion intent through mixed reality head-mounted displays. International Symposium on Robotics Research