ajbouh / substrate

10 stars 3 forks source link

bridge: translate instancing + commands #67

Open progrium opened 9 months ago

progrium commented 9 months ago

Translation is ultimately not an "always on" agent like transcribe. Instead, it allows for instances of language specific translators to be added to a session by commands. Here is a rough API the translate agent can have that can be exposed as commands to the assistant agent as previously discussed in #8:

func (*Agent) AddTranslatorFrom(lang string) error {}
func (*Agent) RemoveTranslatorFrom(lang string) error {}

func (*Agent) AddTranslatorTo(lang string) error {}
func (*Agent) RemoveTranslatorTo(lang string) error {}

Though keep in mind this is session specific. The point was brought up that language names don't have a single correct name and are looked up through codes, so the language strings here will likely need to be passed to a new endpoint of the translator service just to determine the correct language code to use in translation requests. The main idea is the language string here is what would be spoken in a command.

There is a session default language assumed for now to be english but could be determined dynamically at some point based on first user input or some command. Translating from a language would look for and only translate transcriptions not in that language (from spanish looks for spanish, translating to default/english). Translating to a language would translate all messages not in that language (for spanish, any english would be translated to spanish as well). These two modes have some UI implications, but for now can just post the translation after the transcription as before.

There would have to be Commands exposed via CommandProvider interface for these methods, allowing them to be used via bridge assistant voice commands.

See also #8 for discussion on language matching.

ajbouh commented 8 months ago

I'm not convinced that translation shouldn't just always be on. It can make sense to add translation to additional languages beyond English, but I think it should probably just default to always translating English.