Eye Gaze Controlled Robotic Arm for Persons with Severe Speech and Motor Impairment

Vinay Krishna Sharma, Kamalpreet Saluja, Vimal Mollyn, and Pradipta Biswas

In ACM Symposium on Eye Tracking Research and Applications (ETRA '20 Full Papers). Association for Computing Machinery, New York, NY, USA, Article 12, 1–9. DOI:https://doi.org/10.1145/3379155.3391324


This was work done with Dr. Pradipta Biswas and Vinay Sharma from the Indian Institute of Science, Bangalore, during my research internship from June 2019 to  August 2019

People with severe speech and motor impairment (SSMI) lack the fine-grained motor skills required to perform daily activities - often involving picking and placing objects. Backed by recent work, we realized that gaze could be used as an effective input for people with SSMI. This paper describes the design and development of an eye gaze controlled robotic arm that builds upon these ideas. We report two user studies on a pick and drop reachability task. Using the eye gaze controlled interface, participants could complete this pick and drop task at an average duration of less than 15 secs and reach a randomly designated target within 60 secs. It was extremely rewarding to see how the participants reacted to our system - they finally felt a sense of agency. Ultimately, this project led to a publication at ACM ETRA '20