NeonGeckoCom / neon-hana

HTTP Access to DIANA deployed services
Other
0 stars 0 forks source link

[FEAT] Generic translation endpoint to replace Customized LibreTranslate #31

Open NeonDaniel opened 1 month ago

NeonDaniel commented 1 month ago

Objective

Implement a /translate backend to replace calls currently made to a custom LibreTranslate container.

Initial Implementation Requirements

Other Considerations

NeonDaniel commented 1 month ago

Currently, STT and TTS plugins are hosted independent of HANA. I think that either:

The latter option does raise questions about hosting multiple plugins in one backend, but does expose a unified endpoint for all hosted services. Deferring to OVOS server packages means it is less clear where different endpoints exist, but does scale more easily where hosting multiple services is desired.

In any case, the MQ integration Klat uses for translations will likely need to be updated or replaced with an HTTP query

NeonDaniel commented 1 month ago

Relevant Klat integration changes in LibreTranslate

This also includes an example for how services could be attached to MQ as well as the example inputs from Klat that need to be supported

NeonDaniel commented 1 month ago

Currently, STT and TTS plugins are hosted independent of HANA.

Also worth noting that LLMs hosted by Neon are also similar in that they use vLLM and expose their own endpoints.