17 April

What is Sensory Substitution?   Tuesday, April 17, 2012

Posted by : Ryan Clark
Circuit Thumbprint
Sensory substitution can be described as the attempt to evoke the feel of one sensory modality by stimulating the sensory apparatus of another. The classic example is Bach-y-Rita’s work on tactile-visual sensory substitution, published in 1969. Something akin to the feeling of sight was generated through a grid of 20×20 vibrating solenoids, contacting a subject’s skin.

The vibration of each solenoid in the grid corresponded to the brightness of one pixel in a 20×20 image being recorded by a video camera. Subjects were able to move the camera to scan the room and, with practice, were able to “discriminate between individuals, to decide where they are in the room, to describe their posture, movements, and individual characteristics such as height, hair length, presence or absence of glasses and so on.”

An excellent modern example of sensory substitution is the visual-to-auditory system by Peter Meijer called The vOICe. With The vOICe, users hear an auditory representation of the visual information being captured by a camera. You can get a sense for how it works by using this applet or by watching the following videos of the work of Malika Auvray and Amir Amedi.

(Or, to experience something similar, but in the form of a video game, try Z-Rox! Be sure to click “Skip” after the intro or the game doesn’t seem to start.)

We believe that the field of sensory substitution has only narrowly been explored and still has great potential. We plan to be active in this field and invite you to follow us on Twitter or Facebook if you would like to be kept up to date our progress.
Share this post?

Leave a Reply