This wearable device is engineered to deliver personalized navigation cues for visually impaired users, offering information through vibration, sound, or other customizable formats.
The University of Michigan’s Department of Biomedical Engineering is spearheading a groundbreaking National Institutes of Health (NIH)-funded research initiative to address navigational challenges faced by people with vision loss. The project assembles experts from across U-M—BME, the School of Information, Electrical Engineering and Computer Science (EECS), and the Kellogg Eye Center, in collaboration with partners at Western Michigan University—to design, build, and test the Adaptive Vision Assistant (AVA). This wearable device is engineered to deliver personalized navigation cues for visually impaired users, offering information through vibration, sound, or other customizable formats.
AVA’s flexible approach aims to minimize device abandonment—a common problem in assistive technology—by inviting users to select the mode of feedback that best matches their preferences. “People interact with technology in different ways,” explained James Weiland, Professor, Biomedical Engineering, and Ophthalmology and Visual Sciences, and the principal investigator. “Some people prefer audio cues, and others rely on vibrations, so we designed AVA to be customizable. This way, people are more likely to use it regularly and benefit from it.”
Central to the AVA’s development is direct input from individuals with vision loss, such as consultant John Kusku, a teacher and Paralympian who has a progressive retinal disease. Kusku, who represented the United States at the 2016 Summer Paralympics and won a silver medal in goalball, recently visited U-M’s North Campus research lab to test focus group protocols and offer his perspective as a future end-user. “I assisted by working with the team in advance of focus groups by testing the directions in the script and advising on how to recruit group participants,” Kusku said.
“Having progressive retinal disease, I know firsthand how vital it is for assistive technology to be adaptable and easy to use,” he added. “The great opportunity here is to integrate the device to provide navigation aid in the least distractive way possible. It will be exciting to use haptic feedback for navigation.” Haptic technology simulates the sense of touch by applying forces, vibrations, or motions to users, bridging the digital-physical divide. It uses sensors and actuators to provide tactile feedback, improving user experience.
Kusku’s involvement has already proven instrumental, according to Dr. Weiland, reflecting on his recent visit. “John helped us refine our focus group protocols, making sure they are actually tailored to people’s needs. We’re fortunate to work with someone who brings both technical feedback and personal experience.”
In addition to Dr. Weiland, the research team includes:
The project’s next phase will study how AVA can be used in a home environment. Study participants will take AVA home for a few weeks, after a clinic visit for training and customizing AVA’s configuration. This study will add to the information obtained from the focus groups to inform AVA’s user-centered design
From Research to Impact
Wearable assistive technology has been a long-term interest of Weiland’s research group. “One of the main projects in our lab is to create wearable technology to help blind people navigate,” Dr. Weiland said. “We’re not making them see again, but we’re putting forth the idea that if we can help them safely move around their neighborhood, then that will significantly improve overall quality of life.”
A prior collaboration with Ford Motor Company demonstrated how wearable navigation systems can be integrated with vehicle mobility infrastructure. This work, recently published in IEEE Transactions on Intelligent Transportation Systems, showed how cameras at intersections could be used to provide additional information to guide blind pedestrians.
“Our thinking was ‘let’s use autonomous vehicle infrastructure and put a blind user on this same map,’ ” Dr. Weiland explained. “So now the person is using the same information that’s available to autonomous vehicles to know where they are on sidewalks and relative to crosswalks. And that makes our problem somewhat easier if we can essentially take the same strategy as autonomous vehicles, but for a different purpose.”
“We benefited from our Ford Alliance and their willingness to invest in pilot projects,” Weiland said of earlier research. “We learned a lot from testing at M-City, but now our focus is on streamlining the technology into something wearable—something people really WANT to use.”
From Concept to Reality: Simplifying Technology with the Coulter Translational Research Partnership Program
Building on the successes of the autonomous vehicle infrastructure project, Dr. Weiland’s team is taking the next steps toward commercial viability by creating a wearable system that is both user-friendly and product-ready.
“The wearable systems we came up with were very ‘kludgy,’ ” Weiland admitted, referencing past experimental designs that wouldn’t pass muster in real-world usage. “These experimental systems used advanced processors and sensors that enabled high performance, but required a large battery and would be expensive to implement as a product. In the Coulter project, we’re restricting ourselves to a mobile phone with a single connection to a pair of glasses—something that looks like a pair of glasses someone would actually wear.”
In essence, the project aspires to develop a peripheral for a mobile phone—drawing on the ubiquitous nature of smartphones to facilitate user adoption. The simplified design pares down extraneous features associated with fully developed “smart glasses,” concentrating only on essential components such as a camera and microphone.
“The issue with the mobile phone is that if you want to use it as a blind person navigating, you have to hold it up in front of yourself,” Weiland explained. “Instead, placing some of this functionality in the glasses allows users to interact with their world as naturally as possible and frees their hands.”
The focus on a practical system was initiated by support from the UM Coulter Translational Research Partnership Program and a gift to Dr. Weiland’s lab from Roslyn and Brian Chamberlain. Development of the first prototype glasses started in summer 2024 with human testing beginning a year later. This Coulter project reflects U-M’s commitment to translating groundbreaking research into market-ready solutions. “Our goal is to make mobile phones more useful for people with visual impairment, leveraging the technology as a bridge to greater independence,” Dr. Weiland said. U-M are collaborating across departments within the university, as well as with colleagues at Western Michigan University, to advance improved mobility for all.For more information on these and other research projects, visit James Weiland Research Group – Home for the James Weiland Research Group