search

overte-org / overte

Overte open source virtual worlds platform.
https://overte.org/
Other
128 stars 47 forks source link

Script engine audio listener #949

Open ksuprynowicz opened 1 month ago

ksuprynowicz commented 1 month ago

This will allow streaming in-game audio into script engine, where it can later be processed or sent to external speech-to-text system, for example through a websocket allowing things such as subtitles, automatic translation, and LLM-based chatbots that react to speech.

Armored-Dragon commented 2 weeks ago

I would love to have this feature for the automatic translation! No Mans Sky did this really well and it would be impressive to see this feature in a platform like Overte.

ksuprynowicz commented 2 weeks ago

It already can be used for this on Linux :) Together with R3ido we got it running with Collabora's WhisperLive, and this gives instantenous Speech To Text that can then be translated. I think Whisper also has own translation functionality? If you are interested I can share scripts for this.