search

bentoml / OpenLLM

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.
https://bentoml.com
Apache License 2.0
9.92k stars 630 forks source link

bug: missing attribute when trying to deploy llama-2 gptq #282

Closed jaotheboss closed 1 year ago

jaotheboss commented 1 year ago

Describe the bug

I am trying to deploy GPTQ Llama-2 model using OpenLLM. Did not use any configurations, but simply ran the line from the readme documentation.

When tracing the error, it seems like the configurations class in AutoGPTQ is the issue. Not sure if there's a workaround or a fix that can be done in this repo but I'll raise this bug in the other repo as well.

To reproduce

pip install openllm
pip install openllm[gptq]
pip install scipy
openllm start llama --model-id TheBloke/Llama-2-7b-Chat-GPTQ --quantize gptq --device 0

Logs

2023-08-31T04:27:26+0000 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "_service:svc" can be accessed at http://localhost:3000/metrics.
2023-08-31T04:27:26+0000 [INFO] [cli] Starting production HTTP BentoServer from "_service:svc" listening on http://0.0.0.0:3000 (Press CTRL+C to quit)
2023-08-31T04:27:34+0000 [ERROR] [runner:llm-llama-runner:1] An exception occurred while instantiating runner 'llm-llama-runner', see details below:
2023-08-31T04:27:34+0000 [ERROR] [runner:llm-llama-runner:1] Traceback (most recent call last):
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 307, in init_local
    self._set_handle(LocalRunnerRef)
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/bentoml/_internal/runner/runner.py", line 150, in _set_handle
    runner_handle = handle_class(self, *args, **kwargs)
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/bentoml/_internal/runner/runner_handle/local.py", line 27, in __init__
    self._runnable = runner.runnable_class(**runner.runnable_init_params)  # type: ignore
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/openllm/_llm.py", line 1143, in __init__
    if not self.model: raise RuntimeError('Failed to load the model correctly (See traceback above)')
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/openllm/_llm.py", line 795, in model
    model = self.load_model(*self._model_decls, **self._model_attrs)
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/openllm/_llm.py", line 265, in wrapper
    return f(self, *(*model_decls, *decls), **{**model_attrs, **attrs})
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/openllm/serialisation/__init__.py", line 76, in caller
    return getattr(importlib.import_module(f'.{llm.runtime}', __name__), fn)(llm, *args, **kwargs)
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/openllm/serialisation/transformers/__init__.py", line 177, in load_model
    model = infer_autoclass_from_llm(llm, config).from_pretrained(
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 516, in from_pretrained
    return model_class.from_pretrained(
  File "/home/ec2-user/SageMaker/llm_server/.env_openllm/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2470, in from_pretrained
    quantization_method_from_config = config.quantization_config.get(
AttributeError: 'BaseQuantizeConfig' object has no attribute 'get'

Environment

Environment variable

BENTOML_DEBUG=''
BENTOML_QUIET=''
BENTOML_BUNDLE_LOCAL_BUILD=''
BENTOML_DO_NOT_TRACK=''
BENTOML_CONFIG=''
BENTOML_CONFIG_OPTIONS=''
BENTOML_PORT=''
BENTOML_HOST=''
BENTOML_API_WORKERS=''

System information

bentoml: 1.1.4 python: 3.10.8 platform: Linux-5.10.184-175.731.amzn2.x86_64-x86_64-with-glibc2.26 uid_gid: 1000:1000 conda: 22.11.1 in_conda_env: True

conda_packages
```yaml name: JupyterSystemEnv channels: - https://aws-ml-conda-preview.s3.us-west-2.amazonaws.com - nvidia/label/cuda-11.8.0 - pytorch - conda-forge - intel dependencies: - _libgcc_mutex=0.1=conda_forge - _openmp_mutex=4.5=2_gnu - aiobotocore=2.5.4=pyhd8ed1ab_0 - aiohttp=3.8.5=py310h2372a71_0 - aioitertools=0.11.0=pyhd8ed1ab_0 - aiosignal=1.3.1=pyhd8ed1ab_0 - alsa-lib=1.2.3.2=h166bdaf_0 - anyio=3.7.1=pyhd8ed1ab_0 - argon2-cffi=23.1.0=pyhd8ed1ab_0 - argon2-cffi-bindings=21.2.0=py310h5764c6d_3 - arrow=1.2.3=pyhd8ed1ab_0 - asttokens=2.2.1=pyhd8ed1ab_0 - async-timeout=4.0.3=pyhd8ed1ab_0 - atk-1.0=2.38.0=hd4edc92_1 - attrs=23.1.0=pyh71513ae_1 - autovizwidget=0.20.5=pyh1a96a4e_0 - backcall=0.2.0=pyh9f0ad1d_0 - backports=1.0=pyhd8ed1ab_3 - backports.functools_lru_cache=1.6.5=pyhd8ed1ab_0 - beautifulsoup4=4.12.2=pyha770c72_0 - bleach=6.0.0=pyhd8ed1ab_0 - brotlipy=0.7.0=py310h5764c6d_1005 - bzip2=1.0.8=h7f98852_4 - ca-certificates=2023.7.22=hbcca054_0 - cached-property=1.5.2=hd8ed1ab_1 - cached_property=1.5.2=pyha770c72_1 - cairo=1.16.0=ha12eb4b_1010 - certifi=2023.7.22=pyhd8ed1ab_0 - cffi=1.15.1=py310h255011f_3 - charset-normalizer=3.2.0=pyhd8ed1ab_0 - comm=0.1.4=pyhd8ed1ab_0 - cryptography=39.0.0=py310h65dfdc0_0 - dbus=1.13.6=h5008d03_3 - debugpy=1.6.8=py310hc6cd4ac_0 - decorator=5.1.1=pyhd8ed1ab_0 - defusedxml=0.7.1=pyhd8ed1ab_0 - entrypoints=0.4=pyhd8ed1ab_0 - exceptiongroup=1.1.3=pyhd8ed1ab_0 - executing=1.2.0=pyhd8ed1ab_0 - expat=2.5.0=hcb278e6_1 - font-ttf-dejavu-sans-mono=2.37=hab24e00_0 - font-ttf-inconsolata=3.000=h77eed37_0 - font-ttf-source-code-pro=2.038=h77eed37_0 - font-ttf-ubuntu=0.83=hab24e00_0 - fontconfig=2.14.2=h14ed4e7_0 - fonts-conda-ecosystem=1=0 - fonts-conda-forge=1=0 - fqdn=1.5.1=pyhd8ed1ab_0 - freetype=2.12.1=hca18f0e_1 - fribidi=1.0.10=h36c2ea0_0 - frozenlist=1.4.0=py310h2372a71_0 - fsspec=2021.7.0=pyhd8ed1ab_0 - gdk-pixbuf=2.42.8=hff1cb4f_1 - gettext=0.21.1=h27087fc_0 - giflib=5.2.1=h0b41bf4_3 - glib=2.76.4=hfc55251_0 - glib-tools=2.76.4=hfc55251_0 - gmp=6.2.1=h58526e2_0 - graphite2=1.3.13=h58526e2_1001 - graphviz=4.0.0=h5abf519_0 - gst-plugins-base=1.20.2=hcf0ee16_0 - gstreamer=1.20.3=hd4edc92_2 - gtk2=2.24.33=h90689f9_2 - gts=0.7.6=h977cf35_4 - harfbuzz=4.2.0=h40b6f09_0 - hdijupyterutils=0.20.5=pyh1a96a4e_0 - icu=69.1=h9c3ff4c_0 - idna=3.4=pyhd8ed1ab_0 - importlib_resources=6.0.1=pyhd8ed1ab_0 - ipython=8.14.0=pyh41d4057_0 - ipython_genutils=0.2.0=py_1 - ipywidgets=8.1.0=pyhd8ed1ab_0 - isoduration=20.11.0=pyhd8ed1ab_0 - jinja2=3.1.2=pyhd8ed1ab_1 - jmespath=1.0.1=pyhd8ed1ab_0 - jpeg=9e=h0b41bf4_3 - jsonpointer=2.0=py_0 - jsonschema=4.19.0=pyhd8ed1ab_1 - jsonschema-specifications=2023.7.1=pyhd8ed1ab_0 - jsonschema-with-format-nongpl=4.19.0=pyhd8ed1ab_1 - jupyter=1.0.0=py310hff52083_8 - jupyter_console=6.6.3=pyhd8ed1ab_0 - jupyter_core=5.3.1=py310hff52083_0 - jupyter_events=0.7.0=pyhd8ed1ab_2 - jupyter_server=2.7.1=pyhd8ed1ab_0 - jupyter_server_terminals=0.4.4=pyhd8ed1ab_1 - jupyterlab_pygments=0.2.2=pyhd8ed1ab_0 - jupyterlab_widgets=3.0.8=pyhd8ed1ab_0 - keyutils=1.6.1=h166bdaf_0 - krb5=1.20.1=hf9c8cef_0 - lcms2=2.14=h6ed2654_0 - ld_impl_linux-64=2.40=h41732ed_0 - lerc=4.0.0=h27087fc_0 - libabseil=20230125.3=cxx17_h59595ed_0 - libblas=3.9.0=17_linux64_openblas - libcblas=3.9.0=17_linux64_openblas - libclang=13.0.1=default_h7634d5b_3 - libcups=2.3.3=h36d4200_3 - libdeflate=1.14=h166bdaf_0 - libedit=3.1.20191231=he28a2e2_2 - libevent=2.1.10=h9b69904_4 - libexpat=2.5.0=hcb278e6_1 - libffi=3.4.2=h7f98852_5 - libgcc-ng=13.1.0=he5830b7_0 - libgd=2.3.3=h283352f_2 - libgfortran-ng=13.1.0=h69a702a_0 - libgfortran5=13.1.0=h15d22d2_0 - libglib=2.76.4=hebfc3b9_0 - libgomp=13.1.0=he5830b7_0 - libiconv=1.17=h166bdaf_0 - liblapack=3.9.0=17_linux64_openblas - libllvm13=13.0.1=hf817b99_2 - libnsl=2.0.0=h7f98852_0 - libogg=1.3.4=h7f98852_1 - libopenblas=0.3.23=pthreads_h80387f5_0 - libopus=1.3.1=h7f98852_1 - libpng=1.6.39=h753d276_0 - libpq=14.5=h2baec63_5 - libprotobuf=4.23.4=hd1fb520_1 - librsvg=2.54.4=h7abd40a_0 - libsodium=1.0.18=h36c2ea0_1 - libsqlite=3.42.0=h2797004_0 - libstdcxx-ng=13.1.0=hfd8a6a1_0 - libtiff=4.4.0=h82bc61c_5 - libtool=2.4.7=h27087fc_0 - libuuid=2.38.1=h0b41bf4_0 - libuv=1.43.0=h7f98852_0 - libvorbis=1.3.7=h9c3ff4c_0 - libwebp=1.2.4=h522a892_0 - libwebp-base=1.2.4=h166bdaf_0 - libxcb=1.13=h7f98852_1004 - libxkbcommon=1.0.3=he3ba5ed_0 - libxml2=2.9.14=haae042b_4 - libzlib=1.2.13=hd590300_5 - markupsafe=2.1.3=py310h2372a71_0 - matplotlib-inline=0.1.6=pyhd8ed1ab_0 - mistune=2.0.5=pyhd8ed1ab_0 - mock=5.1.0=pyhd8ed1ab_0 - multidict=6.0.4=py310h1fa729e_0 - mysql-common=8.0.32=h14678bc_0 - mysql-libs=8.0.32=h54cf53e_0 - nb_conda=2.2.1=unix_6 - nb_conda_kernels=2.3.1=py310hff52083_2 - nbclassic=0.5.1=pyhd8ed1ab_0 - nbclient=0.8.0=pyhd8ed1ab_0 - nbconvert=7.2.9=pyhd8ed1ab_0 - nbconvert-core=7.2.9=pyhd8ed1ab_0 - nbconvert-pandoc=7.2.9=pyhd8ed1ab_0 - nbformat=5.9.2=pyhd8ed1ab_0 - ncurses=6.4=hcb278e6_0 - nest-asyncio=1.5.6=pyhd8ed1ab_0 - nodejs=14.18.3=h8ca31f7_2 - nose=1.3.7=py_1006 - notebook=6.5.5=pyha770c72_0 - notebook-shim=0.2.3=pyhd8ed1ab_0 - nspr=4.35=h27087fc_0 - nss=3.92=h1d7d5a4_0 - numpy=1.25.2=py310ha4c1d20_0 - openjdk=11.0.1=h516909a_1016 - openssl=1.1.1v=hd590300_0 - overrides=7.4.0=pyhd8ed1ab_0 - packaging=23.1=pyhd8ed1ab_0 - pandas=2.0.3=py310h7cbd5c2_1 - pandoc=3.1.3=h32600fe_0 - pandocfilters=1.5.0=pyhd8ed1ab_0 - pango=1.50.7=hbd2fdc8_0 - parso=0.8.3=pyhd8ed1ab_0 - pcre2=10.40=hc3806b6_0 - pexpect=4.8.0=pyh1a96a4e_2 - pickleshare=0.7.5=py_1003 - pip=23.2.1=pyhd8ed1ab_0 - pixman=0.40.0=h36c2ea0_0 - pkgutil-resolve-name=1.3.10=pyhd8ed1ab_0 - platformdirs=3.10.0=pyhd8ed1ab_0 - plotly=5.16.1=pyhd8ed1ab_0 - prometheus_client=0.17.1=pyhd8ed1ab_0 - prompt-toolkit=3.0.39=pyha770c72_0 - prompt_toolkit=3.0.39=hd8ed1ab_0 - protobuf=4.23.4=py310hb875b13_0 - psutil=5.9.5=py310h1fa729e_0 - psycopg2=2.9.3=py310h5764c6d_1 - pthread-stubs=0.4=h36c2ea0_1001 - ptyprocess=0.7.0=pyhd3deb0d_0 - pure_eval=0.2.2=pyhd8ed1ab_0 - pycparser=2.21=pyhd8ed1ab_0 - pygments=2.16.1=pyhd8ed1ab_0 - pykerberos=1.2.4=py310hd1ceca7_3 - pyopenssl=23.2.0=pyhd8ed1ab_1 - pyqt=5.12.3=py310hff52083_8 - pyqt-impl=5.12.3=py310h1f8e252_8 - pyqt5-sip=4.19.18=py310h122e73d_8 - pyqtchart=5.12=py310hfcd6d55_8 - pyqtwebengine=5.12.1=py310hfcd6d55_8 - pysocks=1.7.1=pyha2e5f31_6 - python=3.10.8=h257c98d_0_cpython - python-dateutil=2.8.2=pyhd8ed1ab_0 - python-fastjsonschema=2.18.0=pyhd8ed1ab_0 - python-json-logger=2.0.7=pyhd8ed1ab_0 - python-tzdata=2023.3=pyhd8ed1ab_0 - python_abi=3.10=3_cp310 - pytz=2023.3=pyhd8ed1ab_0 - pyyaml=6.0.1=py310h2372a71_0 - pyzmq=24.0.1=py310h330234f_1 - qt=5.12.9=h1304e3e_6 - qtconsole=5.4.3=pyhd8ed1ab_0 - qtconsole-base=5.4.3=pyha770c72_0 - qtpy=2.3.1=pyhd8ed1ab_0 - readline=8.2=h8228510_1 - referencing=0.30.2=pyhd8ed1ab_0 - requests=2.31.0=pyhd8ed1ab_0 - requests-kerberos=0.12.0=py310hff52083_4 - rfc3339-validator=0.1.4=pyhd8ed1ab_0 - rfc3986-validator=0.1.1=pyh9f0ad1d_0 - rpds-py=0.9.2=py310hcb5633a_0 - s3fs=2021.7.0=pyhd8ed1ab_0 - s3transfer=0.6.2=pyhd8ed1ab_0 - send2trash=1.8.2=pyh41d4057_0 - setuptools=68.1.2=pyhd8ed1ab_0 - six=1.16.0=pyh6c4a22f_0 - sniffio=1.3.0=pyhd8ed1ab_0 - soupsieve=2.3.2.post1=pyhd8ed1ab_0 - sparkmagic=0.20.5=pyhd8ed1ab_0 - sqlite=3.42.0=h2c6b66d_0 - stack_data=0.6.2=pyhd8ed1ab_0 - tenacity=8.2.3=pyhd8ed1ab_0 - terminado=0.17.1=pyh41d4057_0 - tinycss2=1.2.1=pyhd8ed1ab_0 - tk=8.6.12=h27826a3_0 - tornado=6.3.3=py310h2372a71_0 - traitlets=5.9.0=pyhd8ed1ab_0 - typescript=3.6.4=0 - typing-extensions=4.7.1=hd8ed1ab_0 - typing_extensions=4.7.1=pyha770c72_0 - typing_utils=0.1.0=pyhd8ed1ab_0 - tzdata=2023c=h71feb2d_0 - uri-template=1.3.0=pyhd8ed1ab_0 - wcwidth=0.2.6=pyhd8ed1ab_0 - webcolors=1.13=pyhd8ed1ab_0 - webencodings=0.5.1=py_1 - websocket-client=1.6.2=pyhd8ed1ab_0 - wheel=0.41.2=pyhd8ed1ab_0 - widgetsnbextension=4.0.8=pyhd8ed1ab_0 - wrapt=1.15.0=py310h1fa729e_0 - xorg-fixesproto=5.0=h7f98852_1002 - xorg-inputproto=2.3.2=h7f98852_1002 - xorg-kbproto=1.0.7=h7f98852_1002 - xorg-libice=1.1.1=hd590300_0 - xorg-libsm=1.2.4=h7391055_0 - xorg-libx11=1.8.4=h0b41bf4_0 - xorg-libxau=1.0.11=hd590300_0 - xorg-libxdmcp=1.1.3=h7f98852_0 - xorg-libxext=1.3.4=h0b41bf4_2 - xorg-libxfixes=5.0.3=h7f98852_1004 - xorg-libxi=1.7.10=h7f98852_0 - xorg-libxrender=0.9.10=h7f98852_1003 - xorg-libxtst=1.2.3=h7f98852_1002 - xorg-recordproto=1.14.2=h7f98852_1002 - xorg-renderproto=0.11.1=h7f98852_1002 - xorg-xextproto=7.3.0=h0b41bf4_1003 - xorg-xproto=7.0.31=h7f98852_1007 - xz=5.2.6=h166bdaf_0 - yaml=0.2.5=h7f98852_2 - yarl=1.9.2=py310h2372a71_0 - zeromq=4.3.4=h9c3ff4c_1 - zipp=3.16.2=pyhd8ed1ab_0 - zlib=1.2.13=hd590300_5 - zstd=1.5.2=hfc55251_7 - pip: - aiofiles==22.1.0 - aiosqlite==0.19.0 - astroid==2.15.6 - autopep8==2.0.2 - awscli==1.29.33 - babel==2.12.1 - boto3==1.28.33 - botocore==1.31.33 - cloudpickle==2.2.1 - colorama==0.4.4 - contextlib2==21.6.0 - dill==0.3.7 - docker==6.1.3 - docstring-to-markdown==0.12 - docutils==0.16 - environment-kernels==1.2.0 - gitdb==4.0.10 - gitpython==3.1.32 - google-pasta==0.2.0 - importlib-metadata==6.8.0 - ipykernel==5.5.6 - isort==5.12.0 - jedi==0.18.2 - json5==0.9.14 - jupyter-client==7.4.9 - jupyter-lsp==2.2.0 - jupyter-server-fileid==0.9.0 - jupyter-server-mathjax==0.2.6 - jupyter-server-proxy==3.2.1 - jupyter-server-ydoc==0.8.0 - jupyter-ydoc==0.2.5 - jupyterlab==3.6.5 - jupyterlab-git==0.41.0 - jupyterlab-lsp==4.2.0 - jupyterlab-server==2.24.0 - lazy-object-proxy==1.9.0 - mccabe==0.7.0 - multiprocess==0.70.15 - nbdime==3.2.1 - nbexamples==0.0.0 - pathos==0.3.1 - pid==3.0.4 - pluggy==1.2.0 - pox==0.3.3 - ppft==1.7.6.7 - py4j==0.10.9.5 - pyasn1==0.5.0 - pycodestyle==2.10.0 - pydocstyle==6.3.0 - pyflakes==3.0.1 - pygal==3.0.0 - pylint==2.17.5 - pyspark==3.3.0 - python-lsp-jsonrpc==1.0.0 - python-lsp-server==1.7.4 - pytoolconfig==1.2.5 - rope==1.9.0 - rsa==4.7.2 - sagemaker==2.179.0 - sagemaker-experiments==0.1.45 - sagemaker-nbi-agent==1.0 - sagemaker-pyspark==1.4.5 - schema==0.7.5 - simpervisor==1.0.0 - smdebug-rulesconfig==1.0.1 - smmap==5.0.0 - snowballstemmer==2.2.0 - tblib==1.7.0 - tomli==2.0.1 - tomlkit==0.12.1 - ujson==5.8.0 - urllib3==1.26.14 - y-py==0.6.0 - ypy-websocket==0.8.4 prefix: /home/ec2-user/anaconda3/envs/JupyterSystemEnv ```
pip_packages
``` accelerate==0.22.0 aiohttp==3.8.5 aiosignal==1.3.1 anyio==4.0.0 appdirs==1.4.4 asgiref==3.7.2 async-timeout==4.0.3 attrs==23.1.0 auto-gptq==0.4.2 bentoml==1.1.4 bitsandbytes==0.41.1 build==0.10.0 cattrs==23.1.2 certifi==2023.7.22 charset-normalizer==3.2.0 circus==0.18.0 click==8.1.7 click-option-group==0.5.6 cloudpickle==2.2.1 cmake==3.27.2 coloredlogs==15.0.1 contextlib2==21.6.0 cuda-python==12.2.0 Cython==3.0.2 datasets==2.14.4 deepmerge==1.1.0 Deprecated==1.2.14 dill==0.3.7 exceptiongroup==1.1.3 fastcore==1.5.29 filelock==3.12.3 filetype==1.2.0 frozenlist==1.4.0 fs==2.4.16 fsspec==2023.6.0 ghapi==1.0.4 h11==0.14.0 httpcore==0.17.3 httpx==0.24.1 huggingface-hub==0.16.4 humanfriendly==10.0 idna==3.4 importlib-metadata==6.0.1 inflection==0.5.1 Jinja2==3.1.2 lit==16.0.6 markdown-it-py==3.0.0 MarkupSafe==2.1.3 mdurl==0.1.2 mpmath==1.3.0 multidict==6.0.4 multiprocess==0.70.15 mypy-extensions==1.0.0 networkx==3.1 numpy==1.25.2 nvidia-cublas-cu11==11.10.3.66 nvidia-cuda-cupti-cu11==11.7.101 nvidia-cuda-nvrtc-cu11==11.7.99 nvidia-cuda-runtime-cu11==11.7.99 nvidia-cudnn-cu11==8.5.0.96 nvidia-cufft-cu11==10.9.0.58 nvidia-curand-cu11==10.2.10.91 nvidia-cusolver-cu11==11.4.0.1 nvidia-cusparse-cu11==11.7.4.91 nvidia-nccl-cu11==2.14.3 nvidia-nvtx-cu11==11.7.91 openllm==0.2.27 openllm-client==0.2.27 openllm-core==0.2.27 opentelemetry-api==1.18.0 opentelemetry-instrumentation==0.39b0 opentelemetry-instrumentation-aiohttp-client==0.39b0 opentelemetry-instrumentation-asgi==0.39b0 opentelemetry-sdk==1.18.0 opentelemetry-semantic-conventions==0.39b0 opentelemetry-util-http==0.39b0 optimum==1.12.0 orjson==3.9.5 packaging==23.1 pandas==2.1.0 pathspec==0.11.2 peft==0.5.0 Pillow==10.0.0 pip-requirements-parser==32.0.1 pip-tools==7.3.0 prometheus-client==0.17.1 protobuf==4.24.2 psutil==5.9.5 pyarrow==13.0.0 pydantic==1.10.12 Pygments==2.16.1 pynvml==11.5.0 pyparsing==3.1.1 pyproject_hooks==1.0.0 python-dateutil==2.8.2 python-json-logger==2.0.7 python-multipart==0.0.6 pytz==2023.3 PyYAML==6.0.1 pyzmq==25.1.1 regex==2023.8.8 requests==2.31.0 rich==13.5.2 rouge==1.0.1 safetensors==0.3.3 schema==0.7.5 scipy==1.11.2 sentencepiece==0.1.99 simple-di==0.1.5 six==1.16.0 sniffio==1.3.0 starlette==0.28.0 sympy==1.12 tabulate==0.9.0 tokenizers==0.13.3 tomli==2.0.1 torch==2.0.1 tornado==6.3.3 tqdm==4.66.1 transformers==4.32.1 triton==2.0.0 typing_extensions==4.7.1 tzdata==2023.3 urllib3==2.0.4 uvicorn==0.23.2 watchfiles==0.20.0 wcwidth==0.2.6 wrapt==1.15.0 xxhash==3.3.0 yarl==1.9.2 zipp==3.16.2 ```

System information (Optional)

No response

vipulgote1999 commented 1 year ago

+1

aarnphm commented 1 year ago

This has been fixed for 0.3.0. Please let me know and reopen the issue if you still running into this.