Learning to grasp objects with a hand-worn sensory substitution device

Host Institution: Universidad Autónoma de Madrid

Department that hosts the PhD: Facultad de Psicología

Contact: Dr. Jacobs (use this email contact only to request additional information, do NOT use this email for applications)


Project description

This project focuses on the grasping of objects by blind and blindfolded participants with a vibrotactile sensory substitution device incorporated in a glove. The goal of the project is (a) to reveal which type of vibrotactile information and variability therein is most beneficial as an aid in the learning of the grasping action and (b) to show how variability in joint-angle configurations modifies the relation between vibrotactile information and learning. The research extends previous research with sensory substitution systems on the abdomen, chest and legs, to sensory substitution devices on the hand. Sensory substitution devices on the hand have arguably received less attention of the scientific and rehabilitation communities. The research on sensory substitution is combined with a focus on variability in the conditions of practice, which has been shown to have beneficial effects on learning. Grasping objects is a highly relevant task for the patient population that is addressed in this project (the blind). The task is scientifically interesting because it characterizes the need to study the role of active exploration in learning better than any other task that can be found in the scientific literature on sensory substitution. In the experiments, participants wear a glove with vibrotactile actuators on the fingers that each vibrate with an intensity that is a function of the distance to the nearest object in the pointing direction of the respective fingers: the closer the object, the more intense the vibration. Participants practice the grasping of common objects in different situations. Different functions that relate the object distances to vibration patterns will be explored as well as the interaction of these different types of vibrotactile information with different joint-angle configurations. Initial experiments will be performed with blindfolded participants. The most successful configurations of the device will be tested with blind individuals.


People involved

Dr. Jacobs

Dr. Travieso

Dr. Bongers


Key publications

Lobo, L., Travieso, D., Barrientos, A., & Jacobs, D. M. (2014). Stepping on obstacles with a sensory substitution device on the lower leg: Practice without vision is more beneficial than practice with vision. PLOS ONE, 9(6), e98801.

Lobo, L., Travieso, D., Jacobs, D. M., Rodger, M., & Craig, C. M. (2018). Sensory substitution: Using a vibrotactile device to orient and walk to targets. Journal of Experimental Psychology: Applied, 24, 108-124.

Jacobs, D. M., & Michaels, C. F. (2007). Direct learning. Ecological Psychology, 19, 321-349.


Specific required skills of PhD student

Skill area Experimental Psychology
Language (writing) English
Language (speaking) English, Spanish (preferred or willingness to learn)
Programming Matlab
Statistics ANOVA, multiple regression
Statistical programs SPSS
Background Experimental Psychology

Cognitive Psychology

Human Movement Sciences


Biomedical Engineering



Project specific knowledge Analysis of movement registration data


Project characteristics

Primary focus project Behavioral experiments with adults with and without visual impairments
Methods Developing a glove with vibrotactile actuators, programming and performing learning experiments on grasping movements, analyzing performance and 3D registrations of arm and hand kinematics