search

spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/index.html
Apache License 2.0
3.19k stars 804 forks source link

Use openAiApi but custom request URL #710

Open Dragonchu opened 5 months ago

Dragonchu commented 5 months ago

I'm using a company-specific proxy to interact with OpenAI. Our proxy requires a different URL structure for sending queries, specifically http://gpt-proxy.xx.com/gateway/azure/chat/completions

I can customize the base URL using the spring.ai.openai.base-url configuration, but the OpenAiApi seems to always append /v1/chat/completions to the base URL. This doesn't work with our required URL structure.

Could we have a new configuration property that allows for customization of this appended URI? This way, we could configure our application like this:

spring.ai.openai.base-url=http://gpt-proxy.xx.com/

spring.ai.openai.completions-uri=gateway/azure/chat/completions

This change would make the OpenAiApi more flexible for users with unique URL requirements.

devholic22 commented 5 months ago

@Dragonchu Hello, I think the reason why v1/chat/completions is automatically added is because the chatCompletionEntity,chatCompletionStream method in OpenAiApi.java automatically added uri as /v1/chat/completions. I think it would be possible to overload the functions to receive additional uri. (ex:chatCompletionEntity(uri, ChatCompletionRequest)))

And I'd like to ask the maintainers if this idea is a good way or if it's okay for me to solve it.

스크린샷 2024-05-22 오후 2 06 23

devholic22 commented 5 months ago

Additionally, I think the modification for base-url can be done by overloading the generator for OpenAiConnectionProperties. Can I try this issue?

Amine-Kadi commented 5 months ago

@devholic22 Indeed, it could be possible by overloading the call method. But wouldn't it be better to have it in the app config and have it as an attribute called chatCompletionUri in the class OpenAiParentProperties ? So this property class would have both the base url and the custom wanted uri.

rossbamford-xdesign commented 4 months ago

+1 for this issue, it also prevents Spring AI being used with (e.g.) CloudFlare AI Gateways (which would require the v1 to not be statically appended to the base URL).

Jopo16 commented 2 months ago

+1 for the issue, encojnter the same problemwith company specific URL suffixes. Custom uri in app properties would solve the issue. Has anyone managed to overload the function until then?

ThomasVitale commented 2 months ago

Spring AI 1.0.0-M2 introduced the spring.ai.openai.chat.completions-path property to specify a custom path (by default, it's set to /v1/chat/completions). Would that help in your situations?