search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
134.7k stars 26.94k forks source link

Error loading text generation pipeline: Exception: Python patch version not an integer #20871

Closed morrisalp closed 1 year ago

morrisalp commented 1 year ago

System Info

Platform: Ubuntu 20.04.5, Jupyter Lab 3.5.2, dockerized Python version: 3.8.13

pip freeze output:

absl-py==1.2.0 accelerate==0.15.0 aiohttp==3.8.3 aiosignal==1.3.1 alabaster==0.7.12 anyio==3.6.1 apex==0.1 appdirs==1.4.4 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1660605382950/work async-timeout==4.0.2 attrs==22.1.0 audioread==3.0.0 Babel==2.10.3 backcall @ file:///home/conda/feedstock_root/build_artifacts/backcall_1592338393461/work backports.functools-lru-cache @ file:///home/conda/feedstock_root/build_artifacts/backports.functools_lru_cache_1618230623929/work beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1649463573192/work bitsandbytes==0.35.4 bleach==5.0.1 blis @ file:///home/conda/feedstock_root/build_artifacts/cython-blis_1656314523915/work brotlipy @ file:///home/conda/feedstock_root/build_artifacts/brotlipy_1648854175163/work cachetools==5.2.0 catalogue @ file:///home/conda/feedstock_root/build_artifacts/catalogue_1661366525041/work certifi==2022.9.24 cffi @ file:///home/conda/feedstock_root/build_artifacts/cffi_1656782821535/work chardet @ file:///home/conda/feedstock_root/build_artifacts/chardet_1656142044710/work charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1655906222726/work click @ file:///home/conda/feedstock_root/build_artifacts/click_1651215152883/work cloudpickle==2.2.0 codecov==2.1.12 colorama @ file:///home/conda/feedstock_root/build_artifacts/colorama_1655412516417/work conda-package-handling @ file:///home/conda/feedstock_root/build_artifacts/conda-package-handling_1663583601093/work contourpy==1.0.5 coverage==6.5.0 cryptography @ file:///home/conda/feedstock_root/build_artifacts/cryptography_1665535545125/work cuda-python @ file:///rapids/cuda_python-11.7.0%2B0.g95a2041.dirty-cp38-cp38-linux_x86_64.whl cudf @ file:///rapids/cudf-22.8.0a0%2B304.g6ca81bbc78.dirty-cp38-cp38-linux_x86_64.whl cugraph @ file:///rapids/cugraph-22.8.0a0%2B132.g2daa31b6.dirty-cp38-cp38-linux_x86_64.whl cuml @ file:///rapids/cuml-22.8.0a0%2B52.g73b8d00d0.dirty-cp38-cp38-linux_x86_64.whl cupy-cuda118 @ file:///rapids/cupy_cuda118-11.0.0-cp38-cp38-linux_x86_64.whl cycler==0.11.0 cymem @ file:///home/conda/feedstock_root/build_artifacts/cymem_1636053152744/work Cython==0.29.32 dask @ file:///rapids/dask-2022.7.1-py3-none-any.whl dask-cuda @ file:///rapids/dask_cuda-22.8.0a0%2B36.g9860cad-py3-none-any.whl dask-cudf @ file:///rapids/dask_cudf-22.8.0a0%2B304.g6ca81bbc78.dirty-py3-none-any.whl dataclasses @ file:///home/conda/feedstock_root/build_artifacts/dataclasses_1628958434797/work debugpy==1.6.3 decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work defusedxml==0.7.1 diffusers==0.11.0 distributed @ file:///rapids/distributed-2022.7.1-py3-none-any.whl docutils==0.17.1 entrypoints==0.3 et-xmlfile==1.1.0 executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1665301981797/work expecttest==0.1.3 fastjsonschema==2.16.2 fastrlock==0.8 filelock @ file:///home/conda/feedstock_root/build_artifacts/filelock_1660129891014/work flake8==3.7.9 Flask==2.2.2 fonttools==4.37.4 frozenlist==1.3.3 fsspec==2022.8.2 functorch==0.3.0a0 future==0.18.2 glob2==0.7 google-auth==2.12.0 google-auth-oauthlib==0.4.6 graphsurgeon @ file:///workspace/TensorRT-8.5.0.12/graphsurgeon/graphsurgeon-0.4.6-py2.py3-none-any.whl grpcio==1.49.1 HeapDict==1.0.1 huggingface-hub==0.11.0 hypothesis==4.50.8 idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1642433548627/work imagesize==1.4.1 importlib-metadata==5.0.0 importlib-resources==5.10.0 iniconfig==1.1.1 iopath==0.1.10 ipykernel==6.16.0 ipython @ file:///home/conda/feedstock_root/build_artifacts/ipython_1662481517711/work ipython-genutils==0.2.0 ipywidgets==8.0.2 itsdangerous==2.1.2 jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1659959867326/work Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1654302431367/work joblib==1.2.0 json5==0.9.10 jsonschema==4.16.0 jupyter-core==4.11.1 jupyter-server==1.21.0 jupyter-tensorboard @ git+https://github.com/cliffwoolley/jupyter_tensorboard.git@ffa7e26138b82549453306e06b535a9ac36db17a jupyter_client==7.4.2 jupyterlab==2.3.2 jupyterlab-pygments==0.2.2 jupyterlab-server==1.2.0 jupyterlab-widgets==3.0.3 jupytext==1.14.1 kiwisolver==1.4.4 langcodes @ file:///home/conda/feedstock_root/build_artifacts/langcodes_1636741340529/work libarchive-c @ file:///home/conda/feedstock_root/build_artifacts/python-libarchive-c_1649436017468/work librosa==0.9.2 lightning-utilities==0.4.2 llvmlite==0.39.1 lmdb==1.3.0 locket==1.0.0 Markdown==3.4.1 markdown-it-py==2.1.0 MarkupSafe @ file:///home/conda/feedstock_root/build_artifacts/markupsafe_1648737563195/work matplotlib==3.6.2 matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work mccabe==0.6.1 mdit-py-plugins==0.3.1 mdurl==0.1.2 mistune==2.0.4 mock @ file:///home/conda/feedstock_root/build_artifacts/mock_1648992799371/work msgpack==1.0.4 multidict==6.0.3 murmurhash @ file:///home/conda/feedstock_root/build_artifacts/murmurhash_1636019583024/work nbclassic==0.4.5 nbclient==0.7.0 nbconvert==7.2.1 nbformat==5.7.0 nest-asyncio==1.5.6 networkx==2.6.3 nltk==3.7 notebook==6.4.10 notebook-shim==0.1.0 numba==0.56.2 numpy @ file:///home/conda/feedstock_root/build_artifacts/numpy_1643958805350/work nvidia-dali-cuda110==1.18.0 nvidia-pyindex==1.0.9 nvtx==0.2.5 oauthlib==3.2.1 onnx @ file:///opt/pytorch/pytorch/third_party/onnx openpyxl==3.0.10 packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1637239678211/work pandas==1.4.4 pandocfilters==1.5.0 parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work partd==1.3.0 pathy @ file:///home/conda/feedstock_root/build_artifacts/pathy_1656568808184/work pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1602535608087/work pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work Pillow @ file:///tmp/pillow-simd pkginfo @ file:///home/conda/feedstock_root/build_artifacts/pkginfo_1654782790443/work pkgutil_resolve_name==1.3.10 pluggy==1.0.0 polygraphy==0.42.1 pooch==1.6.0 portalocker==2.5.1 preshed @ file:///home/conda/feedstock_root/build_artifacts/preshed_1636077712344/work prettytable==3.4.1 prometheus-client==0.15.0 prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1662384672173/work protobuf==3.20.1 psutil @ file:///home/conda/feedstock_root/build_artifacts/psutil_1662356143277/work ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work py==1.11.0 pyarrow @ file:///rapids/pyarrow-8.0.0-cp38-cp38-linux_x86_64.whl pyasn1==0.4.8 pyasn1-modules==0.2.8 pybind11==2.10.0 pycocotools @ git+https://github.com/nvidia/cocoapi.git@142b17a358fdb5a31f9d5153d7a9f3f1cd385178#subdirectory=PythonAPI pycodestyle==2.5.0 pycosat @ file:///home/conda/feedstock_root/build_artifacts/pycosat_1649384811940/work pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work pydantic @ file:///home/conda/feedstock_root/build_artifacts/pydantic_1636021149719/work pydot==1.4.2 pyflakes==2.1.1 Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1660666458521/work pylibcugraph @ file:///rapids/pylibcugraph-22.8.0a0%2B132.g2daa31b6.dirty-cp38-cp38-linux_x86_64.whl pynvml==11.4.1 pyOpenSSL @ file:///home/conda/feedstock_root/build_artifacts/pyopenssl_1643496850550/work pyparsing @ file:///home/conda/feedstock_root/build_artifacts/pyparsing_1652235407899/work pyrsistent==0.18.1 PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work pytest==6.2.5 pytest-cov==4.0.0 pytest-pythonpath==0.7.4 python-dateutil==2.8.2 python-hostlist==1.22 python-nvd3==0.15.0 python-slugify==6.1.2 pytorch-lightning==1.8.5.post0 pytorch-quantization==2.1.2 pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1664798238822/work PyYAML @ file:///home/conda/feedstock_root/build_artifacts/pyyaml_1648757091578/work pyzmq==24.0.1 raft @ file:///rapids/raft-22.8.0a0%2B70.g9070c30.dirty-cp38-cp38-linux_x86_64.whl regex==2022.9.13 requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1656534056640/work requests-oauthlib==1.3.1 resampy==0.4.2 revtok @ git+git://github.com/jekbradbury/revtok.git@f1998b72a941d1e5f9578a66dc1c20b01913caab rmm @ file:///rapids/rmm-22.8.0a0%2B62.gf6bf047.dirty-cp38-cp38-linux_x86_64.whl rsa==4.9 ruamel-yaml-conda @ file:///home/conda/feedstock_root/build_artifacts/ruamel_yaml_1653464386701/work sacremoses==0.0.53 safetensors==0.2.6 scikit-learn @ file:///rapids/scikit_learn-0.24.2-cp38-cp38-manylinux2010_x86_64.whl scipy @ file:///home/conda/feedstock_root/build_artifacts/scipy_1619561901336/work seaborn==0.12.1 Send2Trash==1.8.0 shellingham @ file:///home/conda/feedstock_root/build_artifacts/shellingham_1659638615822/work six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work smart-open @ file:///home/conda/feedstock_root/build_artifacts/smart_open_1630238320325/work sniffio==1.3.0 snowballstemmer==2.2.0 sortedcontainers==2.4.0 soundfile==0.11.0 soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1658207591808/work spacy @ file:///home/conda/feedstock_root/build_artifacts/spacy_1644657943105/work spacy-legacy @ file:///home/conda/feedstock_root/build_artifacts/spacy-legacy_1660748275723/work spacy-loggers @ file:///home/conda/feedstock_root/build_artifacts/spacy-loggers_1661365735520/work Sphinx==5.2.3 sphinx-glpi-theme==0.3 sphinx-rtd-theme==1.0.0 sphinxcontrib-applehelp==1.0.2 sphinxcontrib-devhelp==1.0.2 sphinxcontrib-htmlhelp==2.0.0 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 srsly @ file:///home/conda/feedstock_root/build_artifacts/srsly_1638879568141/work stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1664126450622/work tabulate==0.9.0 tblib==1.7.0 tensorboard==2.10.1 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.1 tensorboardX==2.5.1 tensorrt @ file:///workspace/TensorRT-8.5.0.12/python/tensorrt-8.5.0.12-cp38-none-linux_x86_64.whl terminado==0.16.0 text-unidecode==1.3 thinc @ file:///home/conda/feedstock_root/build_artifacts/thinc_1638980259098/work threadpoolctl==3.1.0 tinycss2==1.1.1 tokenizers==0.13.2 toml @ file:///home/conda/feedstock_root/build_artifacts/toml_1604308577558/work tomli==2.0.1 toolz @ file:///home/conda/feedstock_root/build_artifacts/toolz_1657485559105/work torch==1.13.0a0+d0d6b1f torch-tensorrt @ file:///opt/pytorch/torch_tensorrt/py/dist/torch_tensorrt-1.3.0a0-cp38-cp38-linux_x86_64.whl torchinfo==1.7.1 torchmetrics==0.11.0 torchtext==0.11.0a0 torchvision @ file:///opt/pytorch/vision tornado==6.2 tqdm==4.64.1 traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1663005918942/work transformer-engine @ file:///tmp/te_wheel/transformer_engine-0.1.0-cp38-cp38-linux_x86_64.whl transformers==4.25.1 treelite @ file:///rapids/treelite-2.4.0-py3-none-manylinux2014_x86_64.whl treelite-runtime @ file:///rapids/treelite_runtime-2.4.0-py3-none-manylinux2014_x86_64.whl typer @ file:///home/conda/feedstock_root/build_artifacts/typer_1657029164904/work typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1665144421445/work ucx-py @ file:///rapids/ucx_py-0.27.0a0%2B29.ge9e81f8-cp38-cp38-linux_x86_64.whl uff @ file:///workspace/TensorRT-8.5.0.12/uff/uff-0.6.9-py2.py3-none-any.whl urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1658789158161/work wasabi @ file:///home/conda/feedstock_root/build_artifacts/wasabi_1658931821849/work wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1600965781394/work webencodings==0.5.1 websocket-client==1.4.1 Werkzeug==2.2.2 widgetsnbextension==4.0.3 xgboost @ file:///rapids/xgboost-1.6.1-cp38-cp38-linux_x86_64.whl yarl==1.8.2 zict==2.2.0 zipp==3.9.0

Who can help?

@ArthurZucker @younesbelkada @gante @Narsil

Information

Tasks

Reproduction

from transformers import pipeline
generator = pipeline('text-generation', model='gpt2')

Output:

Downloading:   0%|          | 0.00/665 [00:00<?, ?B/s]
Downloading:   0%|          | 0.00/548M [00:00<?, ?B/s]
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In [2], line 1
----> 1 generator = pipeline('text-generation', model='gpt2')

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/pipelines/__init__.py:724, in pipeline(task, model, config, tokenizer, feature_extractor, framework, revision, use_fast, use_auth_token, device, device_map, torch_dtype, trust_remote_code, model_kwargs, pipeline_class, **kwargs)
    720 # Infer the framework from the model
    721 # Forced if framework already defined, inferred if it's None
    722 # Will load the correct model if possible
    723 model_classes = {"tf": targeted_task["tf"], "pt": targeted_task["pt"]}
--> 724 framework, model = infer_framework_load_model(
    725     model,
    726     model_classes=model_classes,
    727     config=config,
    728     framework=framework,
    729     task=task,
    730     **hub_kwargs,
    731     **model_kwargs,
    732 )
    734 model_config = model.config
    735 hub_kwargs["_commit_hash"] = model.config._commit_hash

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/pipelines/base.py:257, in infer_framework_load_model(model, config, model_classes, task, framework, **model_kwargs)
    251     logger.warning(
    252         "Model might be a PyTorch model (ending with `.bin`) but PyTorch is not available. "
    253         "Trying to load the model with Tensorflow."
    254     )
    256 try:
--> 257     model = model_class.from_pretrained(model, **kwargs)
    258     if hasattr(model, "eval"):
    259         model = model.eval()

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py:463, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    461 elif type(config) in cls._model_mapping.keys():
    462     model_class = _get_model_class(config, cls._model_mapping)
--> 463     return model_class.from_pretrained(
    464         pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    465     )
    466 raise ValueError(
    467     f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
    468     f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
    469 )

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/modeling_utils.py:2230, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   2227 if from_pt:
   2228     if not is_sharded and state_dict is None:
   2229         # Time to load the checkpoint
-> 2230         state_dict = load_state_dict(resolved_archive_file)
   2232     # set dtype to instantiate the model under:
   2233     # 1. If torch_dtype is not None, we use that dtype
   2234     # 2. If torch_dtype is "auto", we auto-detect dtype from the loaded state_dict, by checking its first
   2235     #    weights entry that is of a floating type - we assume all floating dtype weights are of the same dtype
   2236     # we also may have config.torch_dtype available, but we won't rely on it till v5
   2237     dtype_orig = None

File /storage/morrisalper/notebooks/envs/notebook_env/lib/python3.8/site-packages/transformers/modeling_utils.py:386, in load_state_dict(checkpoint_file)
    381 """
    382 Reads a PyTorch checkpoint file, returning properly formatted errors if they arise.
    383 """
    384 if checkpoint_file.endswith(".safetensors") and is_safetensors_available():
    385     # Check format of the archive
--> 386     with safe_open(checkpoint_file, framework="pt") as f:
    387         metadata = f.metadata()
    388     if metadata.get("format") not in ["pt", "tf", "flax"]:

Exception: Python patch version not an integer

Expected behavior

Should not output an exception. E.g. this code runs as-is (after pip install transformers) in Google Colab.

Narsil commented 1 year ago

This is very odd.

Could you share maybe a bit more about your environment so we could reproduce ? It seems like the way Python itself is installed is odd (I'm purely inferring from the error message), maybe ?

Is it possible to provide a way to reproduce maybe ? Like a docker image or something ?

It does seem to work on colab so it's hard to know what is wrong with the enviroment. It also seems like there's a mix of conda and pip install which might be at play (both link to different things, so maybe the linker is confused somehow ?) I tried googling your error message but nothing came up..

Codle commented 1 year ago

I met the same problem and fixed it by degrading the transformers version like 4.22.0 or others.

SimonLinnebjerg commented 1 year ago

Experienced the same problem. I also downgraded to make it work.

i dont know what the _commit_hash variable is used for, but removing the line in transformers/pipelines/init.py also seems to work.

this line hub_kwargs["_commit_hash"] = model.config._commit_hash

A fix for this would be very appreciated

Codle commented 1 year ago

I think it was related to this issue. All PyTorch container images of NVIDIA NGC have alpha version tags for PyTorch. cc @Narsil

Narsil commented 1 year ago

Thanks @Codle .

It seems to be indeed the issue. Releasing a new version soon so everyone has access.

Narsil commented 1 year ago

Should be fixed with new version (0.2.8), could you confirm ?

Codle commented 1 year ago

Hi @Narsil, sorry for my late response. After updating safetensors to 0.2.8, it works fine for me.

Narsil commented 1 year ago

Closing this. Thank you for sharing !

morrisalp commented 1 year ago

Updating safetensors solved it for me too. Thanks!

surya-narayanan commented 1 year ago

despite downgrading my safetensors, I get the following error 

Traceback (most recent call last):
  File "/home/suryahari/Vornoi/QA.py", line 5, in <module>
    model = AutoModelForQuestionAnswering.from_pretrained("deepset/roberta-base-squad2")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/suryahari/miniconda3/envs/diffusers/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/suryahari/miniconda3/envs/diffusers/lib/python3.11/site-packages/transformers/modeling_utils.py", line 2629, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/suryahari/miniconda3/envs/diffusers/lib/python3.11/site-packages/transformers/modeling_utils.py", line 447, in load_state_dict
    with safe_open(checkpoint_file, framework="pt") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: No such device (os error 19)