search

espressif / esp-box

The ESP-BOX is a new generation AIoT development platform released by Espressif Systems.
Apache License 2.0
809 stars 191 forks source link

Set OpenAI API Endpoint URL (AEGHB-446) #110

Closed sammcj closed 12 months ago

sammcj commented 1 year ago

We need a way to set the OpenAI API Endpoint URL both in the examples and helper libraries.

This is usually refered to as the "BASE_URL" in other languages and defaults to https://api.openai.com/v1.

This needs to be able to be changed to run against OpenAI compatible APIs with different endpoints, for example I might want to change it to query https://openai-proxy.local/v1.

LHYhorion commented 1 year ago

We need a way to set the OpenAI API Endpoint URL both in the examples and helper libraries.

This is usually refered to as the "BASE_URL" in other languages and defaults to https://api.openai.com/v1.

This needs to be able to be changed to run against OpenAI compatible APIs with different endpoints, for example I might want to change it to query https://openai-proxy.local/v1.

Hello, if you change the URL as you desire, will the format of the POST request change with the URL?

sammcj commented 1 year ago

No, other than the base URL, all other paths on the API are the same. There are several projects that make use of 100% compatible open AI APIs - all you need to do is change the base, your real to your servers address

LHYhorion commented 1 year ago

No, other than the base URL, all other paths on the API are the same. There are several projects that make use of 100% compatible open AI APIs - all you need to do is change the base, your real to your servers address

This is a great idea, can you provide links to the projects you mentioned?

sammcj commented 1 year ago

Yep sure thing, most of the popular LLM servers offer it now.

There’s also nice integration libraries like LiteLLM which basically means any project that uses it automatically gets an openAI compatible API: https://github.com/BerriAI/litellm

The irony that all of this has to happen because openAI isn’t in the slightest bit open is not lost on me.

LHYhorion commented 1 year ago

Yep sure thing, most of the popular LLM servers offer it now.

There’s also nice integration libraries like LiteLLM which basically means any project that uses it automatically gets an openAI compatible API: https://github.com/BerriAI/litellm

The irony that all of this has to happen because openAI isn’t in the slightest bit open is not lost on me.

Okay, thank you very much for your ideas. We will reference the projects you provided to evaluate how to expand the URL to meet your needs.

sammcj commented 1 year ago

Fantastic! Thank you so much.

sammcj commented 1 year ago

I had a hack on the upstream espressif package that provides openAI, I suspect the change might be something like this - https://github.com/espressif/esp-iot-solution/compare/master...sammcj:esp-iot-solution:master

LHYhorion commented 1 year ago

I had a hack on the upstream espressif package that provides openAI, I suspect the change might be something like this - espressif/esp-iot-solution@master...sammcj:esp-iot-solution:master

Certainly, we have seen the modifications you made, and they look appropriate. We will further evaluate, and if there are no issues, we will implement your suggestions. Thank you very much for your advice.