HoloHear

This project was for a Hololens hackathon – in the course of 48 hours, my team of 4 people built an application for the Microsoft Hololens that took spoken words, used the Cortana engine to convert speech to text, and then processed the text and converted that to animations that played in the user’s field of view. As a test application, we had a couple of phrases that were fully animated in ASL, with a character that could be superimposed anywhere the user liked in their environment – such as a small character that would sit on someone’s shoulder, or a larger character that could be standing next to or in front of the person you were speaking with. Additionally, the app could finger-spell any words, using a simple hand animation. The goal was to allow someone who doesn’t know ASL to be able to communicate easily with someone who does.

While this was an interesting project and conceptually worked well, after speaking with potential users of an application like this in the deaf community, we realized that devices as large and visible as the hololens were not desirable, and that most users we spoke to felt that this wasn’t an application they wanted or would use, so we decided to halt any further development.