search

quarkiverse / quarkus-langchain4j

Quarkus Langchain4j extension
https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html
Apache License 2.0
115 stars 64 forks source link

Azure endpoint pattern domain name #660

Closed duchamk closed 4 days ago

duchamk commented 3 weeks ago

Azure provides Open AI not only in the .openai.azure.com domain but also in .azure-api.net. Is it possible to change the pattern? From

static final String AZURE_ENDPOINT_URL_PATTERN = "https://%s.openai.azure.com/openai/deployments/%s";

to e.g.

static final String AZURE_ENDPOINT_URL_PATTERN = "https://%s/openai/deployments/%s";
geoand commented 3 weeks ago

Thanks for reporting.

Is there any benefit from using one or the other?

duchamk commented 3 weeks ago

The domain is used to separate data processed by public AI. This is used by commercial companies and people who do not want their data processed by public artificial intelligence (possible leak). The issue of security of corporate or private data.

geoand commented 3 weeks ago

Thanks.

cc @csotiriou

csotiriou commented 3 weeks ago

I find no reason why this would not change, I don't have access to models of that domain currently, but it seems that @duchamk has access to something like this.

I would propose https://%s.openai.azure.com/openai/deployments/%s to change to https://%s.%s/openai/deployments/%s, where the second argument would be the domain name, which would be configurable, and would be default be openai.azure.com, what do you think, @geoand ?

geoand commented 3 weeks ago

Sounds reasonable to me.

@duchamk would you like to contribute this change?

csotiriou commented 3 weeks ago

@geoand @duchamk I just found that quarkus-langchain4j has the endpoint configuration, which allows the entire model endpoint to be set, and be overriden for every different model.

Perhaps @duchamk can use this property? And if that's the case, do we still need to make the change in the configuration (honest question, I would personally be open to all cases)

duchamk commented 1 week ago

Yes, there is indeed such a configuration variable quarkus.langchain4j.azure-openai.endpoint, but it has a serious drawback. Easy RAG does not work and switching between models has to be done from scratch.

I like @csotiriou's idea with the domain, if possible add the quarkus.langchain4j.azure-openai.domain-name config variable, then you will be able to use the full potential of the extension.

What do you think about it?