New study at Waterloo creates tool to ease web browsing for the visually impaired


Keeping in mind the special needs of the visually impaired, scientists from the University of Waterloo have created a new voice assistant, which will help them to gain web access quickly and effortlessly by means of smart speakers and other such similar devices.

Through the study, the team found an innovative way to merge the elements of voice assistants to screen readers in order to create a tool which will make the process of web searches easier. This tool is called Voice Exploration, Retrieval, and Search (VERSE). It helps in adding reader-like capabilities to virtual assistants, thus also allowing other smart devices such as smartwatches to function as input accelerators to smart speakers.

The VERSE uses voice as its primary input, thus allowing users to give voice commands like “next”, “previous”, “go back” or “go forward”. Furthermore, the VERSE can also be paired with an app that runs on smartphone as well as on smartwatches. These smart devices thus work as input accelerators or as keyboard shortcuts. For instance, by rotating the crown on a smartwatch, it takes VERSE to the next search result, or to the next paragraph or section, as per the navigation mode.

For the study, the team studied the case of 53 visually impaired web searchers. It was observed that half of the participants used voice commands multiple times in a day, along with an array of other smart devices such as speakers, phones, and smart TVs. This data helped the experts in designing the prototype of VERSE.

“At the outset, VERSE resembles other virtual assistants, as the tool allows people to ask a question and have it answered verbally with a word, phrase or passage,” reported Vtyurina, a PhD candidate in Waterloo’s David R. Cheriton School of Computer Science. Reports suggests, she undertook the study during her internship at Microsoft Research.

“VERSE is differentiated by what happens next. If people need more information, they can use VERSE to access other search verticals, for example, news, facts, and related searches, and can visit any article that appears as a search result. For articles, VERSE showcases its screen reader superpowers by allowing people to navigate along words, sentences, paragraphs, or sections,” she adds.