When installing openllm, it installs:
openllm==0.4.44
openllm-client==0.4.44
openllm-core==0.4.44
and when trying to run a model it will ask you to install openllm[vllm]. But when you install the extra vllm, your openllm downgrades to version openllm==0.1.20, which seems pretty old. I mean since you are deprecating pytorch, then at least vllm should be supported for your later versions.. or have I been missing something during installation?
To reproduce
pip install openllm
// check version
pip install openllm[vllm]
// version downgraded
ransformers-cli env
Traceback (most recent call last):
File "/home/iam/.local/bin/transformers-cli", line 5, in
from transformers.commands.transformers_cli import main
ModuleNotFoundError: No module named 'transformers'
Describe the bug
When installing openllm, it installs: openllm==0.4.44 openllm-client==0.4.44 openllm-core==0.4.44
and when trying to run a model it will ask you to install openllm[vllm]. But when you install the extra vllm, your openllm downgrades to version openllm==0.1.20, which seems pretty old. I mean since you are deprecating pytorch, then at least vllm should be supported for your later versions.. or have I been missing something during installation?
To reproduce
pip install openllm // check version pip install openllm[vllm] // version downgraded
Logs
No response
Environment
bentoml:
Environment variable
System information
bentoml: 1.1.11python: 3.12.3platform: Linux-6.8.7-arch1-2-x86_64-with-glibc2.39uid_gid: 1000:1000pip_packages``` accelerate==0.29.3 aiohttp==3.9.5 aiosignal==1.3.1 anyio==4.3.0 appdirs==1.4.4 asgiref==3.8.1 attrs==23.2.0 bentoml==1.1.11 bitsandbytes==0.41.3.post2 build==0.10.0 cattrs==23.1.2 certifi==2024.2.2 charset-normalizer==3.3.2 circus==0.18.0 click==8.1.7 click-option-group==0.5.6 cloudpickle==3.0.0 coloredlogs==15.0.1 contextlib2==21.6.0 cuda-python==12.4.0 datasets==2.19.0 deepmerge==1.1.1 Deprecated==1.2.14 dill==0.3.8 distlib==0.3.8 distro==1.9.0 einops==0.8.0 fastcore==1.5.32 filelock==3.14.0 filetype==1.2.0 frozenlist==1.4.1 fs==2.4.16 fsspec==2024.3.1 ghapi==1.0.5 h11==0.14.0 httpcore==1.0.5 httpx==0.27.0 huggingface-hub==0.22.2 humanfriendly==10.0 idna==3.7 importlib-metadata==6.11.0 inflection==0.5.1 Jinja2==3.1.3 markdown-it-py==3.0.0 MarkupSafe==2.1.5 mdurl==0.1.2 mpmath==1.3.0 multidict==6.0.5 multiprocess==0.70.16 mypy-extensions==1.0.0 networkx==3.3 numpy==1.26.4 nvidia-cublas-cu12==12.1.3.1 nvidia-cuda-cupti-cu12==12.1.105 nvidia-cuda-nvrtc-cu12==12.1.105 nvidia-cuda-runtime-cu12==12.1.105 nvidia-cudnn-cu12==8.9.2.26 nvidia-cufft-cu12==11.0.2.54 nvidia-curand-cu12==10.3.2.106 nvidia-cusolver-cu12==11.4.5.107 nvidia-cusparse-cu12==12.1.0.106 nvidia-ml-py==11.525.150 nvidia-nccl-cu12==2.20.5 nvidia-nvjitlink-cu12==12.4.127 nvidia-nvtx-cu12==12.1.105 openllm==0.4.44 openllm-client==0.4.44 openllm-core==0.4.44 opentelemetry-api==1.20.0 opentelemetry-instrumentation==0.41b0 opentelemetry-instrumentation-aiohttp-client==0.41b0 opentelemetry-instrumentation-asgi==0.41b0 opentelemetry-sdk==1.20.0 opentelemetry-semantic-conventions==0.41b0 opentelemetry-util-http==0.41b0 optimum==1.19.1 orjson==3.10.1 packaging==24.0 pandas==2.2.2 pathspec==0.12.1 pillow==10.3.0 pip-requirements-parser==32.0.1 pip-tools==7.3.0 platformdirs==4.2.1 prometheus_client==0.20.0 protobuf==5.26.1 psutil==5.9.8 pyarrow==16.0.0 pyarrow-hotfix==0.6 pydantic==1.10.15 Pygments==2.17.2 pyparsing==3.1.2 pyproject_hooks==1.1.0 python-dateutil==2.9.0.post0 python-json-logger==2.0.7 python-multipart==0.0.9 pytz==2024.1 PyYAML==6.0.1 pyzmq==26.0.2 regex==2024.4.28 requests==2.31.0 rich==13.7.1 safetensors==0.4.3 schema==0.7.5 scipy==1.13.0 sentencepiece==0.2.0 setuptools==69.5.1 simple-di==0.1.5 six==1.16.0 sniffio==1.3.1 starlette==0.37.2 sympy==1.12 tokenizers==0.19.1 torch==2.3.0 tornado==6.4 tqdm==4.66.2 transformers==4.40.1 typing_extensions==4.11.0 tzdata==2024.1 urllib3==2.2.1 uvicorn==0.29.0 virtualenv==20.26.1 watchfiles==0.21.0 wheel==0.43.0 wrapt==1.16.0 xxhash==3.4.1 yarl==1.9.4 zipp==3.18.1 ```
transformers-cli:
ransformers-cli env Traceback (most recent call last): File "/home/iam/.local/bin/transformers-cli", line 5, in
from transformers.commands.transformers_cli import main
ModuleNotFoundError: No module named 'transformers'
System information (Optional)
OS: Arch Linux