search

lm-sys / FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Apache License 2.0
36.9k stars 4.55k forks source link

openai api server - last two api calls not protected with check_api_key when used with api keys #2518

Open stelterlab opened 1 year ago

stelterlab commented 1 year ago

While playing around with the --api-keys option on the openai_api_server.py I noticed that the last two api calls are not protected with check_api_key. Is this intended?

surak commented 1 year ago

With which model do you use api keys and how so?

stelterlab commented 1 year ago

I'm currently playing around with different popular models like Vicuna, LeoLM and of course Zephyr. I used the Open AI API for testing alternative UIs which where built for ChatGPT (like Chatbot UI and its forks). While looking deeper into the api server I was wondering if there is a reason why these two calls as mentioned above are not protected.

surak commented 1 year ago

I think it's just a matter of not having any kind of key verification for local models. I asked because of this: #2591

stelterlab commented 1 year ago

Verification against GitHub. Nice. I would also welcome support for OAuth2 so that you have a larger choice and could also use Keycloak or Zitadel, for example.

surak commented 1 year ago

I am not a coder, really. That one solves my problem, so I did it and shared it :-) If you know how to do it against OAuth2 (the whole auth thing is a bit daunting to me), I would be more than happy to have a look and help with it. I think it would be useful to me too.

stelterlab commented 1 year ago

I haven't built any applications with OAuth2 support yet, but I'm interested in how this could be used from FastAPI applications or similar. I could give it a try.

stelterlab commented 10 months ago

Just a short update: after testing with Auth0 & Keycloak as oauth2 provider which require normally a user login to obtain a token first, I found another way to protect the API. API gateways. ;-) KrakenD and Kong for example are worth looking at. They come also with features like load balancing and rate limiting.