search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
23.03k stars 3.26k forks source link

Enable mypy type checking #3680

Open simon-mo opened 3 months ago

simon-mo commented 3 months ago

Anything you want to discuss about vllm.

Even though vLLM is type annotated but we did not enable type checking. It would be useful to add it, even incrementally.

rkooo567 commented 3 months ago

The current remaining

[rkooo567] mypy vllm/engine/*.py --follow-imports=skip --config-file pyproject.toml
[rkooo567] mypy vllm/worker/*.py --follow-imports=skip --config-file pyproject.toml
[rkooo567] mypy vllm/spec_decode/*.py --follow-imports=skip --config-file pyproject.toml
[rkooo567] mypy vllm/model_executor/*.py --follow-imports=skip --config-file pyproject.toml
mypy vllm/lora/*.py --follow-imports=skip --config-file pyproject.toml
mypy vlllm/entrypoints/openai/*.py --follow-imports=skip --config=file pyprojet.toml
mypy vllm/transformers_utils/configs/*.py --follow-imports=skip --config=file pyprojet.toml
mypy vllm/transformers_utils/tokenizers/*.py --follow-imports=skip --config=file pyprojet.toml
mypy vllm/distributed/device_communicators/*.py --follow-imports=skip --config=file pyprojet.toml
mypy vllm/attention/backends/*.py --follow-imports=skip --config=file pyprojet.toml
mypy vllm/transformers_utils/tokenizer_group/*.py --follow-imports=skip --config=file pyprojet.toml
mypy vllm/core/block/*.py --follow-imports=skip --config=file pyprojet.toml
mypy vllm/model_executor/layers/*.py --follow-imports=skip --config=file pyprojet.toml
mypy vllm/model_executor/layers/*.py --follow-imports=skip --config=file pyprojet.toml

and misc after

DarkLight1337 commented 1 month ago

For those who are new to the vLLM repo, determining the correct type of each variable is a great way to enhance your understanding of how the code is connected together.

DarkLight1337 commented 2 weeks ago

Quick update on the latest progress:

However, the effectiveness of type checking is still quite minimal because imported types are currently treated as Any due to the setting --follow_imports=skip:

https://github.com/vllm-project/vllm/blob/2be6955a3fd596b33be92a2927f55ee0779a4690/pyproject.toml#L46-L51

My IDE, which uses Pyright basic mode, still flags a lot of errors, showing that there is much work to be done.

Moving forward, we should work on gradually applying --follow_imports=normal for each directory, fixing errors as we go. I recommend going through the directories following a similar order as @rkooo567 did (these search filters should be able to locate the relevant PRs).

youkaichao commented 2 weeks ago

Cool, looking forward to the progress.

rkooo567 commented 1 week ago

This is a great issue to improve the code quality of the repo, so please feel free to work on it if anyone is interested! One example PR in the past https://github.com/vllm-project/vllm/pull/4043