search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
22.35k stars 3.15k forks source link

[Installation]: `ModuleNotFoundError: No module named 'numpy.lib.function_base'` due to NumPy 2.0 release #5587

Open glibg10b opened 3 weeks ago

glibg10b commented 3 weeks ago

Anything you want to discuss about vllm.

Users may see the following error when trying to run vllm:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 26, in <module>
    from vllm.entrypoints.openai.serving_chat import OpenAIServingChat
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/entrypoints/openai/serving_chat.py", line 27, in <module>
    from vllm.model_executor.guided_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/__init__.py", line 6, in <module>
    from vllm.model_executor.guided_decoding.lm_format_enforcer_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/lm_format_enforcer_decoding.py", line 15, in <module>
    from vllm.model_executor.guided_decoding.outlines_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/outlines_decoding.py", line 13, in <module>
    from vllm.model_executor.guided_decoding.outlines_logits_processors import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/outlines_logits_processors.py", line 24, in <module>
    from outlines.fsm.guide import CFGGuide, Generate, Guide, RegexGuide, Write
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/__init__.py", line 2, in <module>
    import outlines.generate
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/generate/__init__.py", line 2, in <module>
    from .cfg import cfg
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/generate/cfg.py", line 5, in <module>
    from outlines.models import OpenAI
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/models/__init__.py", line 14, in <module>
    from .openai import OpenAI, azure_openai, openai
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/models/openai.py", line 9, in <module>
    from outlines.base import vectorize
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/base.py", line 8, in <module>
    from numpy.lib.function_base import (
ModuleNotFoundError: No module named 'numpy.lib.function_base'

This is because numpy recently released a breaking change (2.0.0), and for some reason, vllm's dependencies don't restrict the version based on the major number (which should have been frozen at 1).

5582 works around this, but this hasn't reached me -- I suspect it's because I'm using pip install instead of building from source. A workaround for users like me is to run pip install numpy==1.26.4.

The purpose of this issue is to show up when users search for the above output. Please do not close it until the fix is available to users of pip install.

WoosukKwon commented 3 weeks ago

@glibg10b Thanks for reporting the bug! We are aware of the bug and are currently waiting for either outlines or tensorizer to update its Numpy dependency (e.g., by adding numpy < 2.0.0 in its dependency). If they don't, we will make a new release with the fixed Numpy dependency.

DarkLight1337 commented 2 weeks ago

Fixed by #5582.

Edit: Sorry, just saw your last sentence. Let's rename the title to also include the cause of the problem.