search

brainlid / langchain

Elixir implementation of an AI focused LangChain style framework.
https://hexdocs.pm/langchain/
Other
571 stars 68 forks source link

Fixed google ai default endpoint #118

Closed ljgago closed 2 months ago

ljgago commented 4 months ago

Hello!

In the ChatGoogleAI is wrong the default endpoint.

https://github.com/brainlid/langchain/blob/48add20d097cb0ade21b311a2c5c9bbd61000e93/lib/chat_models/chat_google_ai.ex#L24-L26

https://github.com/brainlid/langchain/blob/48add20d097cb0ade21b311a2c5c9bbd61000e93/lib/chat_models/chat_google_ai.ex#L32-L36

Then, when the final url is geneated, the version is added again:

https://github.com/brainlid/langchain/blob/48add20d097cb0ade21b311a2c5c9bbd61000e93/lib/chat_models/chat_google_ai.ex#L338-L342

This PR only define @default_endpoint with the base url without the api version.

Thanks!

brainlid commented 4 months ago

Thanks @ljgago! I'm not a Google AI user so I rely on others to help support and understand the issues.

Is this related to https://github.com/brainlid/langchain/issues/85?

If so, I think the problem is there are two separate Google APIs that use the same structure/format for their messages and communication, but connections are made differently.

ljgago commented 4 months ago

Hi @brainlid!

I checked and Google has two platforms:

Google AI Gemini API is public and can be used for free (with limitations) with any google account. API REST and the base endpoint is https://generativelanguage.googleapis.com

Google Cloud Vertex AI Gemini API is for Google Cloud platform API REST and the endpoint depends from the region, etc and the api may be different.

I think that ChatGoogleAI module is for Google AI Gemini API and the issue here is that the final url adds the version twice and the call fails.

Thanks!

brainlid commented 4 months ago

Thanks @ljgago for digging into that. What is needed to work well for both systems? Is it just a URL change? Or is there more?

ljgago commented 4 months ago

Here some comparatives:

Google AI Gemini API

Google Cloud Vertex AI Gemini API

The two are very similar with the request model and response model, the main difference is with the url and the auth method.

raulchedrese commented 4 months ago

@brainlid, @ljgago I had a bunch of trouble getting the existing ChatGoogleAI to work with Vertex AI. There are subtle differences in the request formats. For example Vertex seems to have stricter validation on which properties are included in the request and what their values are. I ended up writing a separate module that is working quite well for Gemini through Vertex AI.

I'm happy to contribute some of that code if you think it's helpful but thus far I'm not convinced it is feasible to have a single module handle both APIs.

brainlid commented 4 months ago

I had a bunch of trouble getting the existing ChatGoogleAI to work with Vertex AI. There are subtle differences in the request formats. For example Vertex seems to have stricter validation on which properties are included in the request and what their values are.

Leave it to Google to create two competing and incompatible tools to do the same thing. :roll_eyes:

From what you say @raulchedrese, it sounds like two separate versions may work out best. Can we base the new one on your implementation?

raulchedrese commented 4 months ago

@brainlid yep. I can clean it up and open a PR if you like or feel free to grab the code yourself. Whatever is easiest.

brainlid commented 4 months ago

@raulchedrese I'd really appreciate it as a PR. Also, if you can write some tests around it that would be fantastic.

raulchedrese commented 4 months ago

@brainlid created a PR here.

brainlid commented 3 months ago

@ljgago @raulchedrese contributed a specific VertextAI chat module to handle both the URL difference and other peculiarities between the two implementations.

https://github.com/brainlid/langchain/pull/124

The PR has been merged. Would you please try it out and see if it meets your needs as well?

ljgago commented 3 months ago

@ljgago @raulchedrese contributed a specific VertextAI chat module to handle both the URL difference and other peculiarities between the two implementations.

124

The PR has been merged. Would you please try it out and see if it meets your needs as well?

Hello! I don't have Google Cloud access credentials to try VertexAI, I'm sorry. 😦

brainlid commented 3 months ago

Hello! I don't have Google Cloud access credentials to try VertexAI, I'm sorry. 😦

No worries! My mistake. I thought that what you were asking about.

brainlid commented 2 months ago

The ChatGoogleAI URL issues have been fixed in PR #152 and merged to main.