search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
26.84k stars 3.94k forks source link

[Bug]: 400 Bad Request #4597

Open gaye746560359 opened 4 months ago

gaye746560359 commented 4 months ago

Your current environment

使用docker run --gpus all -v D:\docker\huggingface:/root/.cache/huggingface --env "HUGGING_FACE_HUB_TOKEN=****" -d -p 8000:8000 --ipc=host --name vllm vllm/vllm-openai:latest --model nvidia/Llama3-ChatQA-1.5-8B --api-key vllm-abc123456 --gpu-memory-utilization 0.9部署成功了server端

客户端运行下面的代码报错： from openai import OpenAI

Set OpenAI's API key and API base to use vLLM's API server.

openai_api_key = "vllm-abc123456" openai_api_base = "http://192.168.2.6:8000/v1"

client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, )

chat_response = client.chat.completions.create( model="nvidia/Llama3-ChatQA-1.5-8B", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Tell me a joke."}, ], extra_body = { "guided_choice": ["positive", "negative"] } ).choices[0].message.content print("Chat response:", chat_response)

错误： D:\docker\vllm\venv\Scripts\python.exe D:\docker\vllm\llm_client.py Traceback (most recent call last): File "D:\docker\vllm\llm_client.py", line 14, in chat_response = client.chat.completions.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai_utils_utils.py", line 277, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai\resources\chat\completions.py", line 579, in create return self._post( ^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai_base_client.py", line 1240, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai_base_client.py", line 921, in request return self._request( ^^^^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai_base_client.py", line 1020, in _request raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

400 Bad Request

Bad Request

Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.

🐛 Describe the bug

D:\docker\vllm\venv\Scripts\python.exe D:\docker\vllm\llm_client.py Traceback (most recent call last): File "D:\docker\vllm\llm_client.py", line 14, in chat_response = client.chat.completions.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai_utils_utils.py", line 277, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai\resources\chat\completions.py", line 579, in create return self._post( ^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai_base_client.py", line 1240, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai_base_client.py", line 921, in request return self._request( ^^^^^^^^^^^^^^ File "D:\docker\vllm\venv\Lib\site-packages\openai_base_client.py", line 1020, in _request raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

400 Bad Request

Bad Request

Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.

DarkLight1337 commented 4 months ago

Have you tried using HTTPS instead of HTTP according to the error message?

- openai_api_base = "http://192.168.2.6:8000/v1"
+ openai_api_base = "https://192.168.2.6:8000/v1"
gaye746560359 commented 4 months ago

Have you tried using HTTPS instead of HTTP according to the error message?


- openai_api_base = "http://192.168.2.6:8000/v1"

+ openai_api_base = "https://192.168.2.6:8000/v1"

我是本地部署的，没有https只有http

freespeech4ever commented 4 months ago

I'm seeing a 400 bad request with 0.4.2 (vllm.entrypoints.openai.api_server) that does not occur with 0.4.1 with nothing else changing.... It's strange because if I submit my normal test CURL request, it works fine, but my application using axios returns the 400... Reverting to 0.4.1 and everything works. This has to be a regression.

freespeech4ever commented 2 months ago

The issue I mentioned above still occurs with the latest, 0.5.0-post1. I'm stuck on 0.4.1 to have a functioning openai api server.

dongxiaolong commented 1 month ago

I'm seeing a 400 bad request with 0.4.2 (vllm.entrypoints.openai.api_server) that does not occur with 0.4.1 with nothing else changing.... It's strange because if I submit my normal test CURL request, it works fine, but my application using axios returns the 400... Reverting to 0.4.1 and everything works. This has to be a regression.

I encountered this issue as well. I think it's caused by the current default output being set to streaming. As long as you set 'stream': false, you should be able to get the correct response.

freespeech4ever commented 1 month ago

In my case, it was items in the messages array containing extra fields... date, and _id... This is the core of the breaking change that occurred after 0.4.1. The API is extremely rigid now in validating the input. I've worked around it in my app by stripping out the fields before submission, but not excited about that.

lwdnxu commented 1 week ago

same error , please fix

DarkLight1337 commented 1 week ago

In my case, it was items in the messages array containing extra fields... date, and _id... This is the core of the breaking change that occurred after 0.4.1. The API is extremely rigid now in validating the input. I've worked around it in my app by stripping out the fields before submission, but not excited about that.

This was an intentional change in response to #4339. Sorry for the late reply!

same error , please fix

Please provide more information about this.