Holo-Spot

A Mixed-Reality-based project produced by

Ganlin Zhang *, Deheng Zhang *, Longteng Duan *, Guo Han *

at ETH Zurich.

* Equally contribute to this project

Source code on Github

Our Goals

Implementing and deploying a HoloLens 2 application that enables users to control the Boston Dynamics Spot robot using:

  • Eye tracking

  • Head movements

  • Voice control.

Extension

  • This project would be useful for any mobile robot with an arm.

System Overview

  • Three different modes for different functions.

  • Microsoft Azure Spatial Anchor for co-localization.

  • Robot Operating System (ROS) for communication.

  • Video live stream on Hololens for better visualization

Movement Control

Follow Mode

Spot robot follow the

eye-gaze cursor

Select Mode

Spot robot goes to a

user-specified location

Come Here

Spot robot comes to

the user.

Arm Control

Head Control

Spot arm mimics the pose of

the user's head

Freeze Pose

Fix current arm's pose

restart at different position

Gripper Rotation

Rotate gripper by

tilting head