MIT Visual Microphone: Hearing Through Vision

Can you hear a conversation simply by looking at a person's lips? It sounds like a perfect spy skill to have. Researchers at MIT, Microsoft, and Adobe have developed an algorithm that can reconstruct sound by analyzing vibrations of objects in a video. 


Called the ‘visual microphone’, the technique recovered conversations and music by taping a video and analyzing the vibrations of a an object through soundproof glass. First, a high-speed camera record items (a candy wrapper, a chip bag, or a plant) as they vibrate any sounds in the room. Then, using a previously researched algorithm, they analyze the motions of each item to reconstruct audio information behind each vibration.

The technique is as accurate as the quality of the video, measured by pixels. It can reconstruct sound based on how the edges of those pixels change in color due to sound vibration. It works well both in the same room or from a distance through soundproof glass.

We also recommend watching: "MIT: Finding the Visible in the Invisible" and  "Ishin-Den-Shin: Disney's Audio System Makes Fingertips Sing".

MIT Visual Microphone: Hearing Through Vision

Massachusetts Institute of Technology (MIT) is a private research university in Cambridge, Massachusetts, founded in 1861 in response to the increasing industrialization of the United States. The institute adopted a European polytechnic university model and stressed laboratory instruction in applied science and engineering.