Support `text-multilingual-embedding-002` embedding model

MrCsabaToth commented 2 months ago

Description of the feature request:

https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api lists text-multilingual-embedding-002, however https://ai.google.dev/gemini-api/docs/models/gemini#text-embedding does not. I can successfully instantiate a GenerativeModel, but when I call embedContent like I do with text-embedding-004 it errors out with:

models/text-multilingual-embedding-002 is not found for API version v1beta, or is not supported for embedContent. Call ListModels to see the list of available models and their supported methods.

What problem are you trying to solve with this feature?

I'd like an application to support multiple languages. I inspected embeddings of equivalent sentences on Vertex AI with the text-multilingual-embedding-002 and they were close. Unfortunately text-embedding-004 doesn't seem to be good with the language I tested.

Any other information you'd like to share?

My code:

    final model = GenerativeModel(
      model: 'text-multilingual-embedding-002',
      apiKey: preferences.geminiApiKey,
    );
    final content = Content.text(prompt);
    final embeddingResult = await model.embedContent(content);

davidmigloz commented 2 months ago

Unfortunately, text-multilingual-embedding-002 is only available in VertexAI, not via the Google AI API (which is what this SDK is for). You can use firebase_vertexai package instead.

MrCsabaToth commented 2 months ago

Unfortunately, text-multilingual-embedding-002 is only available in VertexAI, not via the Google AI API (which is what this SDK is for). You can use firebase_vertexai package instead.

Thanks, I'll get to the bottom of this. Interestingly with Firebase Vertex AI I could not use text-multilingual-embedding-002 and even text-embedding-004. It depends on generative-ai-dart package and the object hierarchy is similar (hence I created https://imgflip.com/i/92caxj). https://github.com/firebase/flutterfire/issues/13269

MrCsabaToth commented 2 months ago

@davidmigloz What's the best repository or issue tracker to advocate for inclusion of the multilingual embedding model into Google AI API?

davidmigloz commented 2 months ago

(hence I created https://imgflip.com/i/92caxj)

🤣🤣 Yeah, both APIs are very similar (but not identical), so they reuse the client.

@davidmigloz What's the best repository or issue tracker to advocate for inclusion of the multilingual embedding model into Google AI API?

On the docs page, there's a feedback button in the top right corner, I normally use that channel. Otherwise, there's a gemini-api channel on the Google Developer Community Discord, which is monitored by people on the team. Last but not least, you can always ping Logan on X, he's pretty responsive.

google-gemini / generative-ai-dart