Photo by Ryoji Iwata on Unsplash
Navigating routine daily life is often taken for granted, but it can be quite challenging for the blind. Going into a gym, how would a blind person know what equipment is available? How would she dress fashionably? How about ordering from a menu in a restaurant? With about 1 million Americans legally blind and many more with visual impairment, the problem is not trivial. Be My Eyes is a free mobile app that assists blind people with daily life. It has been a micro-volunteering platform where volunteers help blind users with daily tasks through video calls.
Now this app gets a powerful new tool: GPT-4.
The app has developed Virtual Volunteer based on the image-to-text capability of GPT-4. A user may use the app to send pictures to the system and ask questions about the pictures. GPT-4 enables the system to answer questions promptly. The major advantage of using GPT-4 is that the AI often understands the situation and is able to make context-aware conversations. As an example, if a blind user takes a picture of an item in a fridge, not only can she get the name of the item, but also she can get recipe suggestions and step-by-step guide on how to cook. If the AI fails to address a user’s need, the system will send a real person in to handle the situation. In addition to serving individual users, the system may also be used by businesses that want to improve their accessibility.
Summary
Purpose
To help the blind in daily tasks
Idea
Use a phone camera and GPT-4 to recognize visual information and advise the user with conversations
Further Possibilities
1.Use AI to help the deaf with daily tasks
2. Combine GPT-4 with GPS
3. Have volunteers train GPT-4 with video conferencing
4. Use the same functions for a security camera
5. Build a universal translator
Questions
1. How might AI improve our senses?
2. What are the risks or potential harms for using AI to improve our capabilities?
3. What are all the volunteer organizations that can use AI?
