libretro / RetroArch

Cross-platform, sophisticated frontend for the libretro API. Licensed GPLv3.
http://www.libretro.com
GNU General Public License v3.0
9.82k stars 1.78k forks source link

Add OpenAI API integration for AI Service #16605

Open Mudb0y opened 1 month ago

Mudb0y commented 1 month ago

The OpenAI API has been able to describe images for quite a while now, using GPT4 Vision and later GPT4 Turbo. With the recent addition of GPT4O and the multi-modal capabilities I thought it might be worth investigating adding this feature in to Retroarch natively so you don't have to use external software for descriptions. There's been some more accessibility work being done on the iOS build so I thought I might as well bring this to your attention, as someone who's used GPT4V quite extensively to describe things for me I found it quite useful, even made it act like a sudo screenreader for the 3DS firmware in Citra once so it has it's uses in games for sure.

RaouleMenard commented 1 month ago

GPT4o is close to a professional translator, in all languages you can think of, not just Japanese.

Here's a shot from a NEC PC98 game in Japanese, simple screenshot sent to GPT4o. It's incredible!

Please consider implementing Open AI API to RetroArch. This will be a game changer! pun intended

Screenshot 2024-06-04 at 18-54-54 ChatGPT