Motivation

Over the years, technology has evolved at an ever-increasing rate, affecting all aspects of social life. Following the prominence of disability awareness, developments in the technology world are empowering disabled people by creating better working platforms.

Here, we turn our attention to accessible robot control. Controlling a robot can become quite a challenge for people with physical disabilities. Using a traditional robot controller is often not an option for them. We have tried to put this problem in the context of mixed reality and come up with solutions that provide a smooth and accessible user experience for people with disabilities. We want to leverage the power of mixed reality and HoloLens 2 to develop accessible human-computer interfaces to control or interact with robots.

This project aims to help people with arm or hand amputation to operate the Boston Dynamics Spot robot using HoloLens 2. More specifically, we plan to design and implement a pipeline that enables people to move the robot, control the robot arm, and grasp items by eye tracking, head motion and voice control.