Skip to main content

How We Used VR to Explore What Music Feels Like to a Deaf Person - The New York Times

posted onJanuary 16, 2018
>

Article snippet: Seven years ago, when Rachel Kolb was 20, her friends pitched in to help her learn how to hear music. She was born profoundly deaf and had recently received a cochlear implant to give her partial hearing. “They were so gracious — they made a playlist and annotated it: At this time in the song, it’s this instrument coming in,” she recalled. “So I learned to recognize, oh, that’s a piano, because my friend wrote down, ‘At 35 seconds the piano starts to play.’ ” Rachel is a former Rhodes scholar and current doctoral student whom I met through Peter Catapano, the editor of the Disability series, on the Opinion desk. I work as a senior producer in the VR department at The Times, creating both long- and short-form virtual reality videos. Part of my job is constantly looking for stories that will fit our uniquely experiential medium. Peter introduced me to Rachel after she submitted an essay to him about her experiences of music both before and after receiving the implant. She described music as tactile and visual — not something that you just hear. We thought her story was a great match for the immersive treatment that virtual reality provides. We started to adapt Rachel’s article into a storyboard and quickly settled on a VR piece that would be a mix of animation and live-action, with narration from Rachel. When we met Rachel in person, we were excited to see that she exhibited a natural on-camera presence — vibrantly intelligent and self-possessed. When we worked wit... Link to the full article to read more

Emotional score for this article