search

sgl-project / sglang

SGLang is a fast serving framework for large language models and vision language models.
https://sgl-project.github.io/
Apache License 2.0
5.92k stars 482 forks source link

Invalid API key #466

Closed pseudotensor closed 5 months ago

pseudotensor commented 5 months ago

If I set --api-key , then always get invalid api key from server.

e.g.

python -m sglang.launch_server --model-path lmms-lab/llama3-llava-next-8b --tokenizer-path lmms-lab/llama3-llava-next-8b-tokenizer --port=30000 --host="0.0.0.0" --tp-size=1 --random-seed=1234 --context-length=8192 --api-key=APIKEY

then client (even on same machine) does:

curl -H "Authorization: Bearer APIKEY" http://localhost:30000

always results in 

{"detail":"Invalid API Key"}

I've ensured that there are no other processes around and nothing else using that port. Everything works well if don't set the api key.

alessiodallapiazza commented 5 months ago

Ciao @pseudotensor, to use the API, include the header X-API-Key.

Here’s an example of how to use it with curl:

curl http://localhost:3000/get_model_info -H 'X-API-Key: XXXXXXXXX' -v

Make sure to use the correct X-API-Key header as shown above.

pseudotensor commented 5 months ago

I see, different auth method, thanks!

yur1xpp commented 3 months ago

@alessiodallapiazza is there anyway to start the server that makes it accepts the -H "Authorization: Bearer APIKEY" header?. This X-API-Key header method seems to break tools out there like promptflow.