“All_Aboard” App Effectively Aids Blind and Visually Impaired Commuters in Finding Bus Stops

A team of researchers from Mass Eye and Ear have developed a micro-navigation smartphone app to provide assistance to those who are blind or visually impaired (BVI) in finding their bus stops, and a new study found the success rate of the app was substantially higher than that of Google Maps.

Current GPS systems have sufficient macro-navigation for planning routes using public transportation. However, micro-navigation, such as finding the exact locations of bus stops and destinations, remains an issue for people who are BVI, as GPS-based localization for this is less accurate. To combat this problem, the researchers developed a mobile app called All_Aboard, which is meant to be used in conjunction with mainstream GPS systems and focuses on improving micro-navigation. When a GPS indicates that a BVI user is nearing their destination, that is when All_Aboard should be opened. The app uses the phones’ camera to detect street signs from 30 to 50 feet away. It then uses auditory cues to direct the user towards their destination, with the frequency of the sounds changing as they approach the end point.

The app is powered by artificial intelligence, using a deep learning neural network trained on about 10,000 images of bus stops collected in a given city or region. The app is currently capable of recognizing bus stops in 10 major cities/regions around the world. This video demonstrates how the app works.

In the study, 24 BVI individuals used All_Aboard along with Google Maps to navigate a set route with 10 bus stops at an urban (Boston) and suburban site (Newton, Mass.). The results of the study were measured in terms of localization error and rate of successful localization. Localization error, or gap distance, is defined as the distance between the desired destination and maps marked end point. The rate of successful localization is the probability of getting close enough to the bus stops. The researchers found that in both urban and suburban locations, All_Aboard had a success rate of 93 percent, whereas Google Maps had a 52 percent success rate. Additionally, the average gap distance with Google Maps was 6.62 meters and 1.54 meters with All_Aboard.

GPS accuracy is supposed to be acceptable in suburban areas, according to Luo, who added it was not initially expected that the performance with Google Maps in Newton, Mass. would be so low. Previous research from Luo and his team found this problem may be due to widespread errors in bus stop location mapping data in Google Maps.

“Our findings suggest that the All_Aboard app could help travelers with visual impairments in navigation by accurately detecting the bus stop, and therefore greatly reducing their chance of missing buses due to standing too far from the bus stops,” said Gang Luo, PhD, of the Schepens Eye Research Institute of Mass Eye and Ear. “This study indicates that computer vision-based object recognition capabilities can be used in a complementary way and provide added benefit to purely mapping-based, macro-navigation services in real-world settings.

Read more in TVST.

Authorship: Other co-authors include Shrinivas Pundlik and Prerana Shivshanker of Mass Eye and Ear, and Tim Traut-Savino of the The Carroll Center for the Blind.

Disclosures: The authors declared no potential conflicts of interest with respect to the research. The All_Aboard app evaluated in this study is released to public for free. There is no revenue from app sale or in-app advertisements.

Funding: The All_Aboard app development was funded in part by Microsoft AI4A award.

About Mass Eye and Ear

Massachusetts Eye and Ear, founded in 1824, is an international center for treatment and research and a teaching hospital of Harvard Medical School. A member of Mass General Brigham, Mass Eye and Ear specializes in ophthalmology (eye care) and otolaryngology–head and neck surgery (ear, nose and throat care). Mass Eye and Ear clinicians provide care ranging from the routine to the very complex. Also home to the world's largest community of hearing and vision researchers, Mass Eye and Ear scientists are driven by a mission to discover the basic biology underlying conditions affecting the eyes, ears, nose, throat, head and neck and to develop new treatments and cures. In the 2023–2024 “Best Hospitals Survey,” U.S. News & World Report ranked Mass Eye and Ear #4 in the nation for eye care and #7 for ear, nose and throat care. For more information about life-changing care and research at Mass Eye and Ear, visit our blog, Focus, and follow us on Instagram, Twitter and Facebook.