合并后调用报错：AttributeError: property 'eos_token' of 'ChatGLMTokenizer' object has no setter

[Longleaves]

大佬好，我用合并后的模型，使用原本模型的调用代码结果报错，怎么解决呀？ 代码：

报错： AttributeError: property 'eos_token' of 'ChatGLMTokenizer' object has no setter

[We-IOT]

方便的话，看一下您的transforms的版本 jieba>=0.42.1 ruamel_yaml>=0.18.5 rouge_chinese>=1.0.3 jupyter>=1.0.0 datasets>=2.16.1 peft==0.7.1 transformers>=4.37.2

[We-IOT]

建议pull一下官方的git，保证是新的代码再试一下

[Longleaves]

transformers>=4.37.2

我的transformers是4.38.1，已经是pull官方最新的代码了

能看一下你正在用的transformers的版本吗

[We-IOT]

我也是按照requirements.txt安装的，看出错的地方应该是你合并后模型目录下的tokenization_chatglm.py抛出的错误，

@property
    def eos_token_id(self):
        return self.get_command("<eos>")

我感觉是版本问题引起的，建议pull一个版本，按照requirements.txt安装一遍试一试。

我的pip list 你参考一下：

pip freeze accelerate==0.26.1 aiofiles==23.1.0 aiohttp==3.8.4 aiosignal==1.3.1 altair==4.2.2 annotated-types==0.6.0 anyio==3.6.2 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 arrow==1.3.0 arxiv==2.1.0 asttokens @ file:///opt/conda/conda-bld/asttokens_1646925590279/work astunparse==1.6.3 async-lru==2.0.4 async-timeout==4.0.2 attrs==23.2.0 Babel==2.14.0 backcall @ file:///home/ktietz/src/ci/backcall_1611930011877/work backoff==2.2.1 beautifulsoup4 @ file:///opt/conda/conda-bld/beautifulsoup4_1650462163268/work bitsandbytes==0.42.0 bleach==6.1.0 blinker==1.7.0 blis==0.7.11 boto3==1.34.22 botocore==1.34.22 braceexpand==0.1.7 brotlipy==0.7.0 cachetools==5.3.2 catalogue==2.0.10 certifi @ file:///croot/certifi_1665076670883/work/certifi cffi @ file:///tmp/abs_98z5h56wf8/croots/recipe/cffi_1659598650955/work chardet @ file:///home/builder/ci_310/chardet_1640804867535/work charset-normalizer @ file:///tmp/build/80754af9/charset-normalizer_1630003229654/work click==8.1.3 cloudpathlib==0.16.0 cmake==3.28.3 colorama==0.4.6 comm==0.2.1 conda==22.11.1 conda-build==3.23.3 conda-package-handling @ file:///croot/conda-package-handling_1666940373510/work confection==0.1.4 contourpy==1.0.7 cpm-kernels==1.0.11 cryptography @ file:///croot/cryptography_1665612644927/work cycler==0.11.0 cymem==2.0.8 dataclasses==0.6 dataclasses-json==0.6.4 datasets==2.16.1 debugpy==1.8.0 decorator @ file:///opt/conda/conda-bld/decorator_1643638310831/work deepspeed==0.13.1 defusedxml==0.7.1 dill==0.3.7 distro==1.9.0 dnspython==2.2.1 docstring-parser==0.15 einops==0.7.0 entrypoints==0.4 exceptiongroup==1.0.4 executing @ file:///opt/conda/conda-bld/executing_1646925071911/work expecttest==0.1.4 fastapi==0.109.2 fastjsonschema==2.19.1 feedparser==6.0.10 ffmpy==0.3.0 filelock @ file:///opt/conda/conda-bld/filelock_1647002191454/work flit_core @ file:///opt/conda/conda-bld/flit-core_1644941570762/work/source/flit_core fonttools==4.39.2 fqdn==1.5.1 frozenlist==1.3.3 fsspec==2023.10.0 future==0.18.2 gitdb==4.0.11 GitPython==3.1.41 glob2 @ file:///home/linux1/recipes/ci/glob2_1610991677669/work gql==3.5.0 gradio==3.50.2 gradio_client==0.6.1 graphql-core==3.2.3 greenlet==3.0.3 h11==0.14.0 halo==0.0.31 hjson==3.1.0 httpcore==1.0.2 httpx==0.26.0 huggingface-hub==0.20.3 hypothesis==6.61.0 icetk==0.0.4 idna @ file:///croot/idna_1666125576474/work importlib-metadata==7.0.1 importlib-resources==6.1.1 iniconfig==2.0.0 ipykernel==6.29.2 ipython @ file:///croot/ipython_1670919316550/work ipywidgets==8.1.2 isoduration==20.11.0 jedi @ file:///tmp/build/80754af9/jedi_1644315229345/work jieba==0.42.1 Jinja2==3.1.3 jmespath==1.0.1 joblib==1.3.2 json5==0.9.14 jsonpatch==1.33 jsonpointer==2.4 jsonschema==4.21.1 jsonschema-specifications==2023.12.1 jupyter==1.0.0 jupyter-console==6.6.3 jupyter-events==0.9.0 jupyter-lsp==2.2.2 jupyter_client==8.6.0 jupyter_core==5.7.1 jupyter_server==2.12.5 jupyter_server_terminals==0.5.2 jupyterlab==4.1.0 jupyterlab_pygments==0.3.0 jupyterlab_server==2.25.2 jupyterlab_widgets==3.0.10 kiwisolver==1.4.4 langchain==0.1.6 langchain-community==0.0.19 langchain-core==0.1.22 langchainhub==0.1.14 langcodes==3.3.0 langsmith==0.0.87 latex2mathml==3.77.0 libarchive-c @ file:///tmp/build/80754af9/python-libarchive-c_1617780486945/work linkify-it-py==2.0.0 lit==17.0.6 llmtuner==0.5.1 log-symbols==0.0.14 loguru==0.7.2 Markdown==3.5.2 markdown-it-py==2.2.0 MarkupSafe @ file:///home/builder/ci_310/markupsafe_1640791577099/work marshmallow==3.20.2 matplotlib==3.7.1 matplotlib-inline @ file:///opt/conda/conda-bld/matplotlib-inline_1662014470464/work mdit-py-plugins==0.3.3 mdtex2html==1.3.0 mdurl==0.1.2 mistune==3.0.2 mkl-fft==1.3.1 mkl-random @ file:///home/builder/ci_310/mkl_random_1641843545607/work mkl-service==2.4.0 mpmath==1.2.1 multidict==6.0.4 multiprocess==0.70.15 murmurhash==1.0.10 mypy-extensions==1.0.0 nbclient==0.9.0 nbconvert==7.16.0 nbformat==5.9.2 nest-asyncio==1.6.0 networkx==3.2.1 ninja==1.11.1.1 nltk==3.8.1 notebook==7.0.7 notebook_shim==0.2.3 numpy==1.26.3 nvidia-cublas-cu11==11.10.3.66 nvidia-cublas-cu12==12.1.3.1 nvidia-cuda-cupti-cu11==11.7.101 nvidia-cuda-cupti-cu12==12.1.105 nvidia-cuda-nvrtc-cu11==11.7.99 nvidia-cuda-nvrtc-cu12==12.1.105 nvidia-cuda-runtime-cu11==11.7.99 nvidia-cuda-runtime-cu12==12.1.105 nvidia-cudnn-cu11==8.5.0.96 nvidia-cudnn-cu12==8.9.2.26 nvidia-cufft-cu11==10.9.0.58 nvidia-cufft-cu12==11.0.2.54 nvidia-curand-cu11==10.2.10.91 nvidia-curand-cu12==10.3.2.106 nvidia-cusolver-cu11==11.4.0.1 nvidia-cusolver-cu12==11.4.5.107 nvidia-cusparse-cu11==11.7.4.91 nvidia-cusparse-cu12==12.1.0.106 nvidia-nccl-cu11==2.14.3 nvidia-nccl-cu12==2.19.3 nvidia-nvjitlink-cu12==12.3.101 nvidia-nvtx-cu11==11.7.91 nvidia-nvtx-cu12==12.1.105 openai==1.12.0 orjson==3.8.7 overrides==7.7.0 packaging==23.2 pandas==1.4.4 pandocfilters==1.5.1 parso @ file:///opt/conda/conda-bld/parso_1641458642106/work peft==0.7.1 pexpect @ file:///tmp/build/80754af9/pexpect_1605563209008/work pickleshare @ file:///tmp/build/80754af9/pickleshare_1606932040724/work pillow==10.2.0 pkginfo @ file:///croot/pkginfo_1666725041340/work platformdirs==4.2.0 pluggy==1.4.0 preshed==3.0.9 prometheus-client==0.19.0 prompt-toolkit==3.0.43 protobuf==4.25.2 psutil @ file:///opt/conda/conda-bld/psutil_1656431268089/work ptyprocess @ file:///tmp/build/80754af9/ptyprocess_1609355006118/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl pure-eval @ file:///opt/conda/conda-bld/pure_eval_1646925070566/work py-cpuinfo==9.0.0 pyarrow==14.0.2 pyarrow-hotfix==0.6 pycosat @ file:///croot/pycosat_1666805502580/work pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work pydantic==2.6.1 pydantic_core==2.16.2 pydeck==0.8.1b0 pydub==0.25.1 Pygments==2.17.2 PyJWT==2.8.0 pynvml==11.5.0 pyOpenSSL @ file:///opt/conda/conda-bld/pyopenssl_1643788558760/work pyparsing==3.0.9 pyrsistent==0.19.3 PySocks @ file:///home/builder/ci_310/pysocks_1640793678128/work pytest==8.0.0 python-dateutil==2.8.2 python-etcd==0.4.5 python-json-logger==2.0.7 python-multipart==0.0.6 pytz @ file:///opt/conda/conda-bld/pytz_1654762638606/work PyYAML==6.0 pyzmq==25.1.2 qtconsole==5.5.1 QtPy==2.4.1 referencing==0.33.0 regex==2022.10.31 requests==2.31.0 requests-toolbelt==1.0.0 rfc3339-validator==0.1.4 rfc3986==1.5.0 rfc3986-validator==0.1.1 rich==13.7.0 rouge-chinese==1.0.3 rpds-py==0.17.1 ruamel.yaml==0.18.6 ruamel.yaml.clib==0.2.8 ruff==0.2.1 s3transfer==0.10.0 safetensors==0.4.1 scikit-learn==1.4.0 scipy==1.11.4 seaborn==0.13.1 semantic-version==2.10.0 Send2Trash==1.8.2 sentence-transformers==2.3.1 sentencepiece==0.1.99 sgmllib3k==1.0.0 shellingham==1.5.4 shtab==1.6.5 six @ file:///tmp/build/80754af9/six_1644875935023/work smart-open==6.4.0 smmap==5.0.1 sniffio==1.3.0 sortedcontainers==2.4.0 soupsieve @ file:///croot/soupsieve_1666296392845/work spacy==3.7.2 spacy-legacy==3.0.12 spacy-loggers==1.0.5 spinners==0.0.24 SQLAlchemy==2.0.25 srsly==2.4.8 sse-starlette==2.0.0 stack-data @ file:///opt/conda/conda-bld/stack_data_1646927590127/work starlette==0.36.3 streamlit==1.30.0 SwissArmyTransformer==0.4.11 sympy==1.11.1 tabulate==0.9.0 tenacity==8.2.3 tensorboardX==2.6.2.2 termcolor==2.4.0 terminado==0.18.0 thinc==8.2.2 threadpoolctl==3.2.0 tiktoken==0.6.0 timm==0.9.12 tinycss2==1.2.1 tokenizers==0.15.1 toml @ file:///tmp/build/80754af9/toml_1616166611790/work tomli==2.0.1 tomlkit==0.12.0 toolz @ file:///croot/toolz_1667464077321/work torch==2.2.0 torchelastic==0.2.2 torchtext==0.14.1 torchvision==0.17.0 tornado==6.4 tqdm @ file:///opt/conda/conda-bld/tqdm_1664392687731/work traitlets @ file:///croot/traitlets_1671143879854/work transform==1.2.0 transform-tools==1.0.5 transformers==4.37.2 transforms==0.2.1 trentaml @ file:///app/trentaml-0.0.1.tar.gz triton==2.2.0 trl==0.7.10 typer==0.9.0 types-dataclasses==0.6.6 types-python-dateutil==2.8.19.20240106 types-requests==2.31.0.20240125 typing-inspect==0.9.0 typing_extensions==4.9.0 tyro==0.7.1 tzdata==2023.4 tzlocal==5.2 uc-micro-py==1.0.1 update-checker==0.18.0 uri-template==1.3.0 urllib3==2.2.0 uvicorn==0.27.1 validators==0.22.0 wasabi==1.1.2 watchdog==3.0.0 wcwidth @ file:///Users/ktietz/demo/mc3/conda-bld/wcwidth_1629357192024/work weasel==0.3.4 webcolors==1.13 webdataset==0.2.86 webencodings==0.5.1 websocket-client==1.7.0 websockets==10.4 widgetsnbextension==4.0.10 xformers==0.0.24 xxhash==3.4.1 yarl==1.8.2 zhipuai==2.0.1 zipp==3.17.0

[Longleaves]

谢谢呀，我把chatglm3原模型下的tokenizer_config.json复制替换掉导出模型路径下同名文件即可运行了。

[CloverZhou]

按照复制替换的方法，报错了：NotImplementedError

[CloverZhou]

[RaspberrySun]

@We-IOT @CloverZhou 出现了同样问题，由解决吗？

[sunshinewhy]

@CloverZhou 遇到了同样的问题 请问该怎么解决？ 按照复制替换的方法，报错了：NotImplementedError