search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
26.66k stars 3.91k forks source link

[Performance]: vLLM version issue. #7949

Open zjjznw123 opened 2 weeks ago

zjjznw123 commented 2 weeks ago

Proposal to improve performance

No response

Report of performance regression

No response

Misc discussion on performance

No response

Your current environment (if you think it is necessary)

Is it possible that even with the same version, the code downloaded at different times might be slightly different? For example, if the tag is 0.5.5, could the code downloaded earlier differ slightly from the code downloaded today? Because I noticed that for the same version, the log outputs are different, and the line numbers in the same Python file also differ. So I wanted to ask about this.

Before submitting a new issue...

DarkLight1337 commented 2 weeks ago

How are you downloading/using the vLLM code?

zjjznw123 commented 2 weeks ago

How are you downloading/using the vLLM code?

One was installed via pip, and the other was compiled using a Dockerfile. When running the apiserver.py file in the container, there was a bug. I noticed that the line number of the Python file where the error occurred is different from the line number in the Python file installed via pip

DarkLight1337 commented 2 weeks ago

Make sure the Docker container isn't outdated. For both the pip and Docker installation, can you import vLLM and print its version number?