Facebook introduces technology to help the blind ‘see’
Sitting on her bed with her legs crossed at her ankles, the brunette holds her cellphone in both hands with a big smile on her face. Though she is blind, she is now able to ‘see’ the photos on her Facebook in her mind.
On Tuesday, April 5, Facebook announced a new technology to assist blind people or those with severe visual impairment ‘view’ the pictures on their site. As stated by the company, Automatic Alternative Text is a new development that will give a description of a photo using advancements in object recognition technology.
According to the World Health Organization, 39 million people are blind and 285 million are estimated to be visually impaired worldwide. With people sharing more than 2 billion photos on Facebook a day, the company wanted a way for the world to be more connected, including the visual impaired.
“One more step toward equal access for people who are visually impaired,” said disability consultant/advocate Rustle Rothstein in a Facebook post.
In the past, those with a visual disability could only hear the name of the person who posted the picture. The new tool will literally tell these people what things might be in a photo on Facebook’s iPhone app.
“Now, if they’re using a screen reader on iOS, they’ll hear a richer description of the photo thanks to automatic alt text,” said Facebook. “For example, for a group on the beach, a person using a screen reader on iOS would now hear, ‘this image may contain: three people, smiling, outdoors.”
Led by the head of Facebook’s accessibility team Jeff Wieland, Facebook has been working on AAT for the past 5 years. The purpose of the accessibility team is to help people with disabilities experience Facebook easily, and help the social network achieve its mission of connecting the world.
The team previously built closed captioning for videos and implemented an option to increase the default font size on Facebook for iOS, a feature 10 percent of Facebook users take advantage of. Tuesday, Facebook shared a video showing these users expressing gratitude for the new technology.
“That whole saying a picture is worth a 1000 words, I think it’s true,” said visual impaired user to Facebook. “Unless you have somebody to describe it to you, even having three words, just helps flesh out all the details that I can’t see. That makes me feel included and like I’m a part of it too.”
Facebook’s first blind engineer Matt King developed the prototype for the Automatic Alternative Text. He first tested his technology with a photo of a friend on a bicycle ride in the European countryside. AI says the photo includes grass, trees and clouds near water to give him a better idea of what the picture looked like.
According to Even Grounds Consulting, a blind person needs to load a program called a screen reader in order to use a computer. It allows the person to hear the currently highlighted text on the screen. It also speaks the keys pressed on the keyboard. Without a screen reader, it would be difficult for a person with a visual disability to use a computer.
AAT uses machine learning to recognize the objects in a photo. The learning helps with artificial intelligence by using algorithms to make predictions. A software needs to be shown an object several times, then it will start to identify it.
“Each advancement in object recognition technology means that the Facebook Accessibility team will be able to make technology even more accessible for more people,” said the company. “When people are connected, they can achieve extraordinary things as individuals and as a community. When everyone is connected, we all benefit.”
The new development combined with Facebook uses iPhone’s VoiceOver feature to read the description of the photos out loud to the user. The tech is in its early stages, currently recognizing about 100 words, which will prevent the computer from giving all the details.
AI can recognize if a picture is a selfie, but only identifies objects in categories such as nature, transportation, sports, food and people. For some users, that is enough or at least a good start.
“My dream is that it would also tell me that it includes Christoph with his bike,” King said. “But from my perspective as a blind user, going from essentially zero percent satisfaction from a photo to somewhere in the neighborhood of half, is a huge jump.”
At this time the new tool is only available through the iPhone app, however, Facebook continues to make improvements, and the plan is to add this functionality for other languages as well as Android and the web.
