bentoml / OpenLLM

Run any open-source LLMs, such as Llama, Mistral, as OpenAI compatible API endpoint in the cloud.
https://bentoml.com
Apache License 2.0
10.11k stars 640 forks source link

bug: Pydantic feature deprecated, need downgrade on latest pip version #1076

Closed TownsendSP closed 2 months ago

TownsendSP commented 2 months ago

Describe the bug

After a fresh install of openLLM from pip (python 3.12, fresh venv created with virtualfish, pydantic==2.9.0), openllm hello errored with the following message:

AttributeError: module 'pydantic._internal._std_types_schema' has no attribute 'PREPARE_METHODS'

The solution:

pip install --force-reinstall -v "pydantic==2.7.0"

No other versions of pydantic were tried, but openllm hello functions as expected using pydantic==2.7.0

The full error message:

To reproduce

  1. vf new openllm_venv
  2. pip install openllm
  3. openllm hello

Logs

Error in sys.excepthook:
Traceback (most recent call last):
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/typer/main.py", line 83, in except_hook
    rich_tb = Traceback.from_exception(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/rich/traceback.py", line 340, in from_exception
    rich_traceback = cls.extract(
                     ^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/rich/traceback.py", line 455, in extract
    locals={
           ^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/rich/traceback.py", line 436, in get_locals
    for key, value in iter_locals:
RuntimeError: dictionary changed size during iteration

Original exception was:
Traceback (most recent call last):
  File "/home/tgsp/.virtualenvs/openllmTest/bin/openllm", line 8, in <module>
    sys.exit(app())
             ^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/typer/main.py", line 338, in __call__
    raise e
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/typer/main.py", line 321, in __call__
    return get_command(self)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/typer/core.py", line 728, in main
    return _main(
           ^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/typer/core.py", line 197, in _main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/typer/main.py", line 703, in wrapper
    return callback(**use_params)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/openllm/analytic.py", line 74, in wrapped
    from bentoml._internal.utils.analytics import track
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/bentoml/__init__.py", line 210, in <module>
    import _bentoml_sdk
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/_bentoml_sdk/__init__.py", line 11, in <module>
    add_custom_preparers()
  File "/home/tgsp/.virtualenvs/openllmTest/lib/python3.12/site-packages/_bentoml_sdk/_pydantic.py", line 194, in add_custom_preparers
    *_std_types_schema.PREPARE_METHODS,
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'pydantic._internal._std_types_schema' has no attribute 'PREPARE_METHODS'

### Environment

#### Environment variable
```bash
BENTOML_DEBUG=''
BENTOML_QUIET=''
BENTOML_BUNDLE_LOCAL_BUILD=''
BENTOML_DO_NOT_TRACK=''
BENTOML_CONFIG=''
BENTOML_CONFIG_OPTIONS=''
BENTOML_PORT=''
BENTOML_HOST=''
BENTOML_API_WORKERS=''

System information

bentoml: 1.3.3 python: 3.12.4 platform: Linux-6.10.5-zen1-1-zen-x86_64-with-glibc2.40 uid_gid: 1000:1000

pip_packages
``` aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 aiosqlite==0.20.0 annotated-types==0.7.0 anyio==4.4.0 appdirs==1.4.4 asgiref==3.8.1 attrs==24.2.0 bentoml==1.3.3 cattrs==23.1.2 certifi==2024.8.30 circus==0.18.0 click==8.1.7 click-option-group==0.5.6 cloudpickle==3.0.0 deepmerge==2.0 deprecated==1.2.14 distro==1.9.0 dulwich==0.22.1 frozenlist==1.4.1 fs==2.4.16 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 httpx-ws==0.6.0 idna==3.8 importlib-metadata==6.11.0 inflection==0.5.1 inquirerpy==0.3.4 jinja2==3.1.4 jiter==0.5.0 markdown-it-py==3.0.0 markupsafe==2.1.5 mdurl==0.1.2 multidict==6.0.5 numpy==2.1.1 nvidia-ml-py==11.525.150 openai==1.41.0 openllm==0.6.10 opentelemetry-api==1.20.0 opentelemetry-instrumentation==0.41b0 opentelemetry-instrumentation-aiohttp-client==0.41b0 opentelemetry-instrumentation-asgi==0.41b0 opentelemetry-sdk==1.20.0 opentelemetry-semantic-conventions==0.41b0 opentelemetry-util-http==0.41b0 packaging==24.1 packaging==24.1 pathlib==1.0.1 pathspec==0.12.1 pfzy==0.3.4 pip==24.1.2 pip-requirements-parser==32.0.1 prometheus-client==0.20.0 prompt-toolkit==3.0.36 psutil==6.0.0 pyaml==24.7.0 pydantic==2.7.0 pydantic-core==2.18.1 pygments==2.18.0 pygments==2.18.0 pyparsing==3.1.4 python-dateutil==2.9.0.post0 python-json-logger==2.0.7 python-multipart==0.0.9 pyyaml==6.0.2 pyzmq==26.2.0 questionary==2.0.1 rich==13.8.0 schema==0.7.7 setuptools==74.1.2 shellingham==1.5.4 simple-di==0.1.5 six==1.16.0 sniffio==1.3.1 starlette==0.38.4 tabulate==0.9.0 tomli-w==1.0.0 tornado==6.4.1 tqdm==4.66.5 typer==0.12.5 typing-extensions==4.12.2 tzdata==2024.1 urllib3==2.2.2 uv==0.4.5 uvicorn==0.30.6 watchfiles==0.24.0 wcwidth==0.2.13 wrapt==1.16.0 wsproto==1.2.0 yarl==1.9.11 zipp==3.20.1 ```

System information (Optional)

OS: Garuda Linux x86_64 Host: MS-7B51 (1.0) Kernel: Linux 6.10.5-zen1-1-zen Uptime: 1 day, 18 hours, 39 mins Packages: 2278 (pacman)[stable] Shell: fish 3.7.1 Display (D32x-D1): 1920x1080 @ 60 Hz in 32″ [External] DE: KDE Plasma 6.1.4 WM: KWin (X11) Cursor: Sweet (24px) Terminal: tmux 3.4 CPU: Intel(R) Core(TM) i7-9700K (8) @ 5.00 GHz GPU 1: AMD Radeon RX 580 Series [Discrete] GPU 2: NVIDIA GeForce RTX 2070 SUPER [Discrete] Memory: 6.37 GiB / 31.27 GiB (20%) Swap: 7.51 GiB / 31.27 GiB (24%)

aarnphm commented 2 months ago

This has been fixed on latest bentoml version (1.3.4post1 afaik)