Add Gemini Pro/Ultra option? (higher BLEU scores compared to GPT4)

[m1ndy]

Hi, would you be interested in allowing to use DeepL or Gemini Pro/Ultra via the same API key flow?

Here's a report about the translation comparison: https://slator.com/google-says-gemini-beats-openai-in-automatic-speech-translation/

I am trying to translate a whole library of subs to various languages and would love it if this could support multiple LLM APIs to allow for easy generation and in-domain comparison

[machinewrapped]

Hi m1ndy, definitely interested - I've already started working towards it, but supporting multiple providers requires quite a big redesign of the system. Hopefully I will have something up and running in the next few weeks though.

[machinewrapped]

Support for Google Gemini is now available in the latest (pre-)release, v0.6.1. https://github.com/machinewrapped/gpt-subtrans/releases/tag/v0.6.1

My initial tests look very promising, Gemini 1.0 Pro seems to produce slightly more natural translations than GPT 3.5, and is basically free to use if you're in a supported region. Please try it out and let me know how you find it.

[m1ndy]

So excited to try this! I got Gemini 1.5 Pro access recently, so would be curious to know if that model is better than 1.0. Though the access is not allowed through api key just yet... only through AI Studio interface, so we'll have to wait. (I don't see it in the list of models when I list curl https://generativelanguage.googleapis.com/v1beta/models?key=$API_KEY)

Any idea if DeepL support is of interest? Though might not be required anymore if "Øpen"AI/Google suffice for most

[machinewrapped]

Great I look forward to your report!

I just updated the release by the way, I was getting random GUI crashes and I think it's because I had downgraded PySide6 due to Mac issues, but I remember it being unstable so I've updated it again for the Windows package. Just redownload the zip file if you encounter the same issues.

I'm not sure if there's anything to be gained by supporting DeepL, my sense was that it has been superseded by LLMs with their contextual understanding and world knowledge, but it's purely supposition on my part. AFAIK it doesn't support following custom instructions so I'm not sure there'd be any advantage to using GPT-Subtrans vs. uploading the SRT file to https://www.deepsubtitles.com/.

[yuxi-liu-wired]

The Claude family would also be useful to support.

[machinewrapped]

Yeah, I watched AI Explained's video on Claude 3 this morning, it looks really interesting! No time this week, but I will definitely be investigating integration when I have chance.

[machinewrapped]

Closing this issue as complete, Gemini support is now in an official release.

[m1ndy]

I'm sorry to ping here again, but can you enable Vertex AI API for Gemini instead of the AI Studio API?

Gemini 1.5 Pro and 1.0 Ultra are only on Vertex AI API endpoints.

I believe these should be a quick URL change, not much other differences.

[machinewrapped]

Hi, I think it's more complicated than that!

I actually started implementing Gemini support via Vertex AI because of the region restrictions on the generative AI endpoint, but it seemed to involve installing the full Google AI SDK (that includes things like support for training your own models) and configuring a project on Vertex AI to get a project ID as well as an API key - it seems more targeted towards business users and back-end systems rather than end users.

If you can find a link that shows an easier way to access it let me know though, and I'll take a look :-)

[machinewrapped]

@yuxi-liu-wired Claude support is now available as a pre-release version: https://github.com/machinewrapped/gpt-subtrans/releases/tag/v0.6.4