HUAWEI has created an app, that allows blind people to ‘see’ or experience emotions on the faces of people they are speaking to using the power of artificial intelligence (AI).
The Facing Emotions app utilises the powerful camera and AI of the HUAWEI Mate 20 Pro, to translate seven universal human emotions, into seven unique sounds. Making it possible for the blind and visually impaired to have a richer experience and understanding when communicating with others.
Facing Emotions was created to push the possibilities of AI technology. By asking “how can AI help the blind and visually impaired to experience or ‘see’ more of the world?” HUAWEI were able to find a unique new application for the power of AI. By teaching it to analyse facial expressions and emotions, and translate them into sounds, they created a new language for the blind to ‘see’ emotions like never before.
Using the back cameras, the application scans the face of the person with whom the blind person is talking, and identify elements of the face: like eyes, nose, brows and mouth, along with their position in relation to each other. AI then processes the identified emotion into a defined sound heard on the phone (or in the earphone the blind person is wearing). All of this happens in a real time and in offline mode. It is only possible thanks to a powerful processor and AI, which Mate 20 Pro is equipped with.
The project was an opportunity to use technology to try and help find a solution to a real world challenge experienced by the blind. Facing Emotions was developed with the support of the blind community and charitable organisations. HUAWEI partnered with the Polish Blind Association and created a key group of blind testers which were involved in every key step of the app development process.
The mobile manufacturer interviewed blind community representatives about their needs and based on those insights they were developing solutions which were consulted with and tested by blind or visually impaired people. It was their input that helped shape the final user experience, as well as its functionalities, colours, and sounds.
“I was amazed by how quickly things could be implemented in the app. It is amazing, and the holder is a great success,” said Monika Woznialis, blind test team Member.
A key component of this process was the sounds themselves. They were created by blind composer Tomasz Bilecki – known among other things for the Invisible Exhibition – to capture each of the seven emotions. The sounds had to be short, concise, simple and non-invasive for the user and their environment. His unique perspective on the nature of sound allowed him to convey the emotions to other visually impaired people. Sound samples were then tested by HUAWEI’s team of blind consultants, who rated them as easy to remember and understand.
“The sounds I wanted to create needed to be specific for each emotion, and not distract users from their conversations,” said Tomasz.
During the app development process, HUAWEI learned that visually impaired people would ideally love a hands free device solution so they have easy access to it with their hands often used for navigating or holding a cane. So as part of the campaign, award-winning designer Janek Kochanski developed a special 3D printed phone holder for the Mate 20 Pro that makes it easier for visually impaired people to use the app and their phone in general.
“For me, as a designer, function is foremost, and the form often results from that. First, the usage requirements have to be satisfied, only then can the aesthetic and visual values be met. The phone that the holder is designed for, the Huawei Mate 20 Pro, is very modern and stylish, but so is the phone case,” said Janek.
To show how much difference the Facing Emotions app can make in people’s lives HUAWEI created a short documentary film. The film featured a key moment in the lives of three blind and visually impaired people, when understanding the emotions displayed on their loved one’s face were incredibly important for them.
The people featured – Mirjana from Sweden, Robert from Poland, and Mustafa from Turkey, were found through HUAWEI’s partnership with local blind NGO’s in several countries searching for real people who would like to use the app and be part of the project. Their stories and their reactions were incredibly moving, making this powerful short film something that will stay with viewers for a long time.
“The Facing Emotions campaign embodies HUAWEI’s mission to to bring new technology to everyone, especially those that need it most. Our new flagship HUAWEI Mate 20 Pro advanced artificial intelligence has given us a unique opportunity to let us help blind people “see” emotions,” said Mr. Wang, President, CEE&Nordic Region, Huawei Consumer Business Group.
Source: HUAWEI
HUAWEI’s new app uses AI to help the blind ‘see’ emotions through the power of sound added by newsroom on
View all posts by newsroom →
You must be logged in to post a comment Login