When running the "Loading with Transformers" example code, an error "Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 78 column 3" will occur

When running the "Loading with Transformers" example code, an error "Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 78 column 3" will occur.

Error result:

Example code: https://github.com/xverse-ai/XVERSE-MoE-A4.2B/blob/main/README_EN.md#loading-with-transformers.

local python pacakges:

Package Version accelerate 0.29.3 aiofiles 23.2.1 altair 5.3.0 annotated-types 0.6.0 anyio 4.3.0 attrs 23.2.0 certifi 2024.2.2 charset-normalizer 3.3.2 click 8.1.7 contourpy 1.2.1 cycler 0.12.1 exceptiongroup 1.2.1 fastapi 0.110.2 ffmpy 0.3.2 filelock 3.13.4 fonttools 4.51.0 fsspec 2024.3.1 gradio 4.27.0 gradio_client 0.15.1 h11 0.14.0 httpcore 1.0.5 httpx 0.27.0 huggingface-hub 0.22.2 idna 3.7 importlib_resources 6.4.0 Jinja2 3.1.3 jsonschema 4.21.1 jsonschema-specifications 2023.12.1 kiwisolver 1.4.5 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.8.4 mdurl 0.1.2 mpmath 1.3.0 networkx 3.3 numpy 1.26.4 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 8.9.2.26 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu12 2.19.3 nvidia-nvjitlink-cu12 12.4.127 nvidia-nvtx-cu12 12.1.105 orjson 3.10.1 packaging 24.0 pandas 2.2.2 pillow 10.3.0 pip 24.0 psutil 5.9.8 pydantic 2.7.1 pydantic_core 2.18.2 pydub 0.25.1 Pygments 2.17.2 pyparsing 3.1.2 python-dateutil 2.9.0.post0 python-multipart 0.0.9 pytz 2024.1 PyYAML 6.0.1 referencing 0.35.0 regex 2024.4.16 requests 2.31.0 rich 13.7.1 rpds-py 0.18.0 ruff 0.4.1 safetensors 0.4.3 semantic-version 2.10.0 setuptools 69.5.1 shellingham 1.5.4 six 1.16.0 sniffio 1.3.1 starlette 0.37.2 sympy 1.12 tokenizers 0.19.1 tomlkit 0.12.0 toolz 0.12.1 torch 2.2.2 tqdm 4.66.2 transformers 4.40.0 triton 2.2.0 typer 0.12.3 typing_extensions 4.11.0 tzdata 2024.1 urllib3 2.2.1 uvicorn 0.29.0 websockets 11.0.3 wheel 0.43.0

xverse-ai / XVERSE-MoE-A4.2B

When running the "Loading with Transformers" example code, an error "Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 78 column 3" will occur #2