OpenAI had previously announced plans to offer real-time visual support through ChatGPT. However, when this feature was still not available in May 2024, I developed a simple tool to enable it, utilizing OpenAI's latest model, GPT-4o.
This integration through Siri Shortcuts allows AI to assist in any visual context. For instance, you can press the action button while using a camera app on your phone to learn more about your surroundings.
If you're browsing a website on your computer, a click on the app in the dock will provide various optimization tips to improve the page's performance. Additionally, if you're drawing something on your iPad, you can have the AI guess what it is.
The integration is fully customizable and can be optimized to enhance its value based on individual needs.
Credits
Concept
Mark van Nieuwenhoven
