macbookpro m1量化baichuan2-13b失败

[mairongchao]

命令：python3 chatglm_cpp/convert.py -i baichuan-inc/Baichuan2-13B-Chat -t q4_0 -o baichuan-ggml.bin 环境：

packages in environment at /opt/homebrew/Caskroom/miniconda/base/envs/pytorch:

#

Name Version Build Channel

accelerate 0.24.1 pyhd8ed1ab_0 conda-forge aiofiles 22.1.0 py312hca03da5_0 aiohttp 3.9.0 py312he37b823_0 conda-forge aiosignal 1.2.0 pyhd3eb1b0_0 altair 5.0.1 py312hca03da5_0 anyio 3.7.1 pyhd8ed1ab_0 conda-forge appnope 0.1.3 py312hca03da5_1001 argon2-cffi 21.3.0 pyhd3eb1b0_0 argon2-cffi-bindings 21.2.0 py312h80987f9_0 asttokens 2.0.5 pyhd3eb1b0_0 attrs 23.1.0 py312hca03da5_0 aws-c-auth 0.7.7 h886c30d_1 conda-forge aws-c-cal 0.6.9 hea61927_1 conda-forge aws-c-common 0.9.8 h93a5062_0 conda-forge aws-c-compression 0.2.17 hea61927_6 conda-forge aws-c-event-stream 0.3.2 h0574dc0_7 conda-forge aws-c-http 0.7.14 h90b1786_2 conda-forge aws-c-io 0.13.36 he1b4ce3_0 conda-forge aws-c-mqtt 0.9.10 h8d54690_1 conda-forge aws-c-s3 0.4.1 ha5b923c_0 conda-forge aws-c-sdkutils 0.1.12 hea61927_5 conda-forge aws-checksums 0.1.17 hea61927_5 conda-forge aws-crt-cpp 0.24.7 hba4ac3b_6 conda-forge aws-sdk-cpp 1.11.182 h31542fa_7 conda-forge backcall 0.2.0 pyhd3eb1b0_0 beautifulsoup4 4.12.2 py312hca03da5_0 bitsandbytes 0.41.2.post2 pypi_0 pypi blas 1.0 openblas bleach 4.1.0 pyhd3eb1b0_0 blinker 1.6.2 py312hca03da5_0 bottleneck 1.3.5 py312ha86b861_0 brotli 1.0.7 hc377ac9_0 brotli-python 1.0.9 py312h313beb8_7 bzip2 1.0.8 h93a5062_5 conda-forge c-ares 1.22.1 h93a5062_0 conda-forge ca-certificates 2023.11.17 hf0a4a13_0 conda-forge cachetools 4.2.2 pyhd3eb1b0_0 certifi 2023.7.22 py312hca03da5_1 cffi 1.16.0 py312h80987f9_0 charset-normalizer 2.0.4 pyhd3eb1b0_0 chatglm-cpp 0.3.0 pypi_0 pypi click 8.1.7 py312hca03da5_0 colorama 0.4.6 py312hca03da5_0 comm 0.1.2 py312hca03da5_0 contourpy 1.0.5 py312h48ca7d4_0 cryptography 41.0.3 py312hd4332d6_0 cycler 0.11.0 pyhd3eb1b0_0 dataclasses 0.8 pyh6d0b6a4_7 datasets 2.14.7 pyhd8ed1ab_0 conda-forge debugpy 1.6.7 py312h313beb8_0 decorator 5.1.1 pyhd3eb1b0_0 defusedxml 0.7.1 pyhd3eb1b0_0 dill 0.3.7 py312hca03da5_0 entrypoints 0.4 py312hca03da5_0 exceptiongroup 1.0.4 py312hca03da5_0 executing 0.8.3 pyhd3eb1b0_0 fastapi 0.104.1 pyhd8ed1ab_0 conda-forge ffmpeg 4.2.2 h04105a8_0 ffmpy 0.3.0 pyhb6f538c_0 conda-forge filelock 3.9.0 py312hca03da5_0 fonttools 4.25.0 pyhd3eb1b0_0 freetype 2.12.1 h1192e45_0 frozenlist 1.4.0 py312h80987f9_0 fsspec 2023.9.2 py312hca03da5_0 gettext 0.21.1 h0186832_0 conda-forge gflags 2.2.2 hc377ac9_0 giflib 5.2.1 h80987f9_3 gitdb 4.0.7 pyhd3eb1b0_0 gitpython 3.1.37 py312hca03da5_0 glog 0.6.0 h6da1cb0_0 conda-forge gmp 6.2.1 hc377ac9_3 gnutls 3.6.15 h887c41c_0 gradio 3.50.2 pyhd8ed1ab_0 conda-forge gradio-client 0.6.1 pyhd8ed1ab_0 conda-forge h11 0.12.0 pyhd3eb1b0_0 h2 4.0.0 py312hca03da5_3 hpack 4.0.0 py_0 httpcore 0.15.0 py312hca03da5_0 httpx 0.23.0 py312hca03da5_0 huggingface_hub 0.17.3 py312hca03da5_0 hyperframe 6.0.1 pyhd3eb1b0_0 icu 73.2 hc8870d7_0 conda-forge idna 3.4 py312hca03da5_0 importlib-metadata 6.0.0 py312hca03da5_0 importlib_metadata 6.0.0 hd3eb1b0_0 importlib_resources 5.2.0 pyhd3eb1b0_1 ipykernel 6.26.0 pyh3cd1d5f_0 conda-forge ipython 8.15.0 py312hca03da5_0 ipython_genutils 0.2.0 pyhd3eb1b0_1 ipywidgets 7.6.5 pyhd3eb1b0_1 jedi 0.18.1 py312hca03da5_1 jinja2 3.1.2 py312hca03da5_0 joblib 1.2.0 py312hca03da5_0 jpeg 9e h80987f9_1 jsonschema 4.17.3 py312hca03da5_0 jupyter_client 7.1.2 pyhd3eb1b0_0 jupyter_core 5.5.0 py312h81bd7bf_0 conda-forge jupyter_server 1.13.5 pyhd3eb1b0_0 jupyterlab_pygments 0.1.2 py_0 jupyterlab_widgets 3.0.5 py312hca03da5_0 kiwisolver 1.4.4 py312h313beb8_0 krb5 1.20.1 hf3e1bf2_1 lame 3.100 h1a28f6b_0 latex2mathml 3.76.0 pypi_0 pypi lcms2 2.12 hba8e193_0 lerc 3.0 hc377ac9_0 libabseil 20230802.1 cxx17_h13dd4ca_0 conda-forge libarrow 14.0.1 h63c4d5d_4_cpu conda-forge libarrow-acero 14.0.1 had9dd58_4_cpu conda-forge libarrow-dataset 14.0.1 had9dd58_4_cpu conda-forge libarrow-flight 14.0.1 h1011bfc_4_cpu conda-forge libarrow-flight-sql 14.0.1 h660fe36_4_cpu conda-forge libarrow-gandiva 14.0.1 h2b96968_4_cpu conda-forge libarrow-substrait 14.0.1 h594d712_4_cpu conda-forge libblas 3.9.0 1_h2ec9a88_netlib conda-forge libbrotlicommon 1.1.0 hb547adb_1 conda-forge libbrotlidec 1.1.0 hb547adb_1 conda-forge libbrotlienc 1.1.0 hb547adb_1 conda-forge libcblas 3.9.0 1_h9886b1c_netlib libcrc32c 1.1.2 hc377ac9_0 libcurl 8.4.0 h3e2b118_0 libcxx 16.0.6 h4653b0c_0 conda-forge libdeflate 1.8 h1a28f6b_5 libedit 3.1.20221030 h80987f9_0 libev 4.33 h1a28f6b_1 libevent 2.1.12 h02f6b3c_1 libexpat 2.5.0 hb7217d7_1 conda-forge libffi 3.4.2 h3422bc3_5 conda-forge libgfortran 5.0.0 11_3_0_hca03da5_28 libgfortran5 11.3.0 h009349e_28 libgoogle-cloud 2.12.0 hfb399a7_4 conda-forge libgrpc 1.59.3 hbcf6334_0 conda-forge libiconv 1.17 he4db4b2_0 conda-forge libidn2 2.3.4 h80987f9_0 liblapack 3.9.0 1_h9886b1c_netlib libllvm15 15.0.7 h504e6bf_3 conda-forge libnghttp2 1.57.0 h62f6fdd_0 libopenblas 0.3.21 h269037a_0 libopus 1.3 h1a28f6b_1 libparquet 14.0.1 heaab74a_4_cpu conda-forge libpng 1.6.39 h80987f9_0 libprotobuf 4.24.4 hc9861d8_0 conda-forge libre2-11 2023.06.02 h1753957_0 conda-forge libsodium 1.0.18 h1a28f6b_0 libsqlite 3.44.2 h091b4b1_0 conda-forge libssh2 1.10.0 h02f6b3c_2 libtasn1 4.19.0 h80987f9_0 libthrift 0.19.0 h026a170_1 conda-forge libtiff 4.4.0 h2fd578a_2 libunistring 0.9.10 h1a28f6b_0 libutf8proc 2.8.0 h1a8c8d9_0 conda-forge libuv 1.46.0 hb547adb_0 conda-forge libvpx 1.10.0 hc377ac9_0 libwebp 1.3.2 ha3663a8_0 libwebp-base 1.3.2 h80987f9_0 libxml2 2.11.6 h0d0cfa8_0 conda-forge libzlib 1.2.13 h53f4e23_5 conda-forge llvm-openmp 17.0.5 hcd81f8e_0 conda-forge loguru 0.7.2 py312h81bd7bf_1 conda-forge lz4-c 1.9.4 h313beb8_0 markdown 3.5.1 pypi_0 pypi markdown-it-py 2.2.0 py312hca03da5_1 markupsafe 2.1.1 py312h80987f9_0 matplotlib-base 3.8.0 py312hd77ebd4_0 matplotlib-inline 0.1.6 py312hca03da5_0 mdtex2html 1.2.0 pypi_0 pypi mdurl 0.1.0 py312hca03da5_0 mistune 2.0.4 py312hca03da5_0 mpmath 1.3.0 py312hca03da5_0 multidict 6.0.4 py312h670c8ac_1 conda-forge multiprocess 0.70.15 py312hca03da5_0 munkres 1.1.4 py_0 nbclassic 1.0.0 pyhb4ecaf3_1 conda-forge nbclient 0.5.11 pyhd3eb1b0_0 nbconvert 7.11.0 pyhd8ed1ab_0 conda-forge nbconvert-core 7.11.0 pyhd8ed1ab_0 conda-forge nbconvert-pandoc 7.11.0 pyhd8ed1ab_0 conda-forge nbformat 5.1.3 pyhd3eb1b0_0 ncurses 6.4 h463b476_2 conda-forge nest-asyncio 1.5.6 py312hca03da5_0 nettle 3.7.3 h84b5d62_1 networkx 3.1 py312hca03da5_0 nomkl 3.0 0 notebook 6.5.4 pyha770c72_0 conda-forge notebook-shim 0.2.3 pyhd8ed1ab_0 conda-forge numexpr 2.8.7 py312h0f3ea24_0 numpy 1.26.0 py312h7f4fdc5_0 numpy-base 1.26.0 py312he047099_0 openh264 1.8.0 h98b2900_0 openjpeg 2.3.0 h7a6adac_2 openssl 3.2.0 h0d3ecfb_0 conda-forge orc 1.9.2 h7c018df_0 conda-forge orjson 3.9.10 py312h37b1423_0 conda-forge packaging 23.1 py312hca03da5_0 pandas 2.1.1 py312hd77ebd4_0 pandoc 3.1.3 hce30654_0 conda-forge pandocfilters 1.5.0 pyhd3eb1b0_0 parso 0.8.3 pyhd3eb1b0_0 pexpect 4.8.0 pyhd3eb1b0_3 pickleshare 0.7.5 pyhd3eb1b0_1003 pillow 10.0.1 py312h3b245a6_0 pip 23.3.1 pyhd8ed1ab_0 conda-forge platformdirs 4.0.0 pyhd8ed1ab_0 conda-forge prometheus_client 0.14.1 py312hca03da5_0 prompt-toolkit 3.0.36 py312hca03da5_0 protobuf 4.24.4 py312h71bcfcd_0 conda-forge psutil 5.9.0 py312h80987f9_0 ptyprocess 0.7.0 pyhd3eb1b0_2 pure_eval 0.2.2 pyhd3eb1b0_0 pyarrow 14.0.1 py312h01ffab7_4_cpu conda-forge pyarrow-hotfix 0.6 pyhd8ed1ab_0 conda-forge pycparser 2.21 pyhd3eb1b0_0 pydantic 1.10.12 py312h80987f9_1 pydeck 0.8.0 pyhd8ed1ab_0 conda-forge pydub 0.25.1 pyhd8ed1ab_0 conda-forge pygments 2.15.1 py312hca03da5_1 pyopenssl 23.2.0 py312hca03da5_0 pyparsing 3.0.9 py312hca03da5_0 pyrsistent 0.18.0 py312h80987f9_0 pysocks 1.7.1 py312hca03da5_0 python 3.12.0 h47c9636_0_cpython conda-forge python-dateutil 2.8.2 pyhd3eb1b0_0 python-multipart 0.0.6 py312hca03da5_0 python-tzdata 2023.3 pyhd3eb1b0_0 python-xxhash 2.0.2 py312h80987f9_1 python_abi 3.12 4_cp312 conda-forge pytorch 2.1.0 cpu_generic_py312h28b293b_0 conda-forge pytz 2023.3.post1 py312hca03da5_0 pyyaml 6.0.1 py312h80987f9_0 pyzmq 25.1.0 py312h313beb8_0 re2 2023.06.02 h6135d0a_0 conda-forge readline 8.2 h92ec313_1 conda-forge regex 2023.10.3 py312h80987f9_0 requests 2.31.0 py312hca03da5_0 rfc3986 1.4.0 pyhd3eb1b0_0 rich 13.3.5 py312hca03da5_1 sacremoses 0.0.43 pyhd3eb1b0_0 safetensors 0.3.3 py312h0002256_1 conda-forge semantic_version 2.8.5 pyhd3eb1b0_0 send2trash 1.8.0 pyhd3eb1b0_1 sentencepiece 0.1.99 py312h48ca7d4_0 setuptools 68.2.2 pyhd8ed1ab_0 conda-forge six 1.16.0 pyhd3eb1b0_1 sleef 3.5.1 h80987f9_2 smmap 4.0.0 pyhd3eb1b0_0 snappy 1.1.10 h17c5cce_0 conda-forge sniffio 1.2.0 py312hca03da5_1 soupsieve 2.5 py312hca03da5_0 sse-starlette 1.6.5 pyhd8ed1ab_0 conda-forge stack_data 0.2.0 pyhd3eb1b0_0 starlette 0.27.0 pyhd8ed1ab_0 conda-forge streamlit 1.28.2 pyhd8ed1ab_0 conda-forge sympy 1.12 pyh04b8f61_3 conda-forge tabulate 0.9.0 pypi_0 pypi tenacity 8.2.2 py312hca03da5_1 terminado 0.17.1 py312hca03da5_0 tinycss2 1.2.1 py312hca03da5_0 tk 8.6.13 h5083fa2_1 conda-forge tokenizers 0.15.0 pypi_0 pypi toml 0.10.2 pyhd3eb1b0_0 toolz 0.12.0 py312hca03da5_0 tornado 6.3.3 py312h80987f9_0 tqdm 4.63.0 pyhd3eb1b0_0 traitlets 5.7.1 py312hca03da5_0 transformers 4.35.2 pypi_0 pypi typing-extensions 4.8.0 hd8ed1ab_0 conda-forge typing_extensions 4.8.0 pyha770c72_0 conda-forge tzdata 2023c h71feb2d_0 conda-forge tzlocal 2.1 py312hca03da5_1 urllib3 1.26.18 py312hca03da5_0 uvicorn 0.24.0 py312h81bd7bf_0 conda-forge validators 0.18.2 pyhd3eb1b0_0 watchdog 2.1.6 py312h80987f9_0 wcwidth 0.2.5 pyhd3eb1b0_0 webencodings 0.5.1 py312hca03da5_2 websocket-client 0.58.0 py312hca03da5_4 websockets 10.4 py312h80987f9_1 wheel 0.41.3 pyhd8ed1ab_0 conda-forge widgetsnbextension 3.5.1 pyh9f0ad1d_3 conda-forge x264 1!152.20180806 h1a28f6b_0 xxhash 0.8.0 h1a28f6b_3 xz 5.2.6 h57fd34a_0 conda-forge yaml 0.2.5 h1a28f6b_0 yarl 1.9.2 py312he37b823_1 conda-forge zeromq 4.3.4 hc377ac9_0 zipp 3.11.0 py312hca03da5_0 zlib 1.2.13 h53f4e23_5 conda-forge zstd 1.5.5 h4f39d0f_0 conda-forge 报错： (pytorch) mrc@mrcdeMacBook-Pro chatglm.cpp % python3 chatglm_cpp/convert.py -i baichuan-inc/Baichuan2-13B-Chat -t q4_0 -o baichuan-ggml.bin

Traceback (most recent call last): File "/Volumes/mydisk/model/chatglm.cpp/chatglm_cpp/convert.py", line 543, in main() File "/Volumes/mydisk/model/chatglm.cpp/chatglm_cpp/convert.py", line 537, in main convert(f, args.model_name_or_path, dtype=args.type) File "/Volumes/mydisk/model/chatglm.cpp/chatglm_cpp/convert.py", line 469, in convert tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 755, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2024, in from_pretrained return cls._from_pretrained( ^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2256, in _from_pretrained tokenizer = cls(init_inputs, **init_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mrc/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan2-13B-Chat/8f6e343d545c503b91429582231d1d354dac2740/tokenization_baichuan.py", line 71, in init super().init( File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/tokenization_utils.py", line 367, in init self._add_tokens( File "/opt/homebrew/Caskroom/miniconda/base/envs/pytorch/lib/python3.12/site-packages/transformers/tokenization_utils.py", line 467, in _add_tokens current_vocab = self.get_vocab().copy() ^^^^^^^^^^^^^^^^ File "/Users/mrc/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan2-13B-Chat/8f6e343d545c503b91429582231d1d354dac2740/tokenization_baichuan.py", line 105, in get_vocab vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)} ^^^^^^^^^^^^^^^ File "/Users/mrc/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan2-13B-Chat/8f6e343d545c503b91429582231d1d354dac2740/tokenization_baichuan.py", line 101, in vocab_size return self.sp_model.get_piece_size() ^^^^^^^^^^^^^ AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'

[li-plus]

最近上游 transformers 更新有不兼容的改动，可以降级到 4.33 试试：

pip install transformers~=4.33.0

[mairongchao]

pip install transformers~=4.33.0

pip install bitsandbytes pip install scipy 运行上面命令，已经解决问题