peiyanyang
commented
10 months ago 包版本：
Package Version
absl-py 1.4.0 accelerate 0.26.0 aiohttp 3.9.1 aiosignal 1.3.1 airllm 2.8.3 anyio 3.6.2 argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 arrow 1.2.3 asttokens 2.2.1 async-timeout 4.0.3 attrs 22.2.0 Babel 2.11.0 backcall 0.2.0 beautifulsoup4 4.11.1 bitsandbytes 0.42.0 bleach 6.0.0 brotlipy 0.7.0 cachetools 5.3.0 certifi 2022.12.7 cffi 1.15.1 charset-normalizer 2.0.4 coloredlogs 15.0.1 comm 0.1.2 conda 22.11.1 conda-content-trust 0.1.3 conda-package-handling 1.9.0 contourpy 1.0.7 cryptography 38.0.1 cycler 0.11.0 datasets 2.16.1 debugpy 1.6.6 decorator 5.1.1 defusedxml 0.7.1 dill 0.3.7 einops 0.7.0 executing 1.2.0 fastjsonschema 2.16.2 filelock 3.13.1 flash-attn 2.4.2 fonttools 4.38.0 fqdn 1.5.1 frozenlist 1.4.1 fsspec 2023.10.0 google-auth 2.16.0 google-auth-oauthlib 0.4.6 grpcio 1.51.1 huggingface-hub 0.20.2 humanfriendly 10.0 idna 3.4 ipykernel 6.20.2 ipython 8.9.0 ipython-genutils 0.2.0 ipywidgets 8.0.4 isoduration 20.11.0 jedi 0.18.2 Jinja2 3.1.2 json5 0.9.11 jsonpointer 2.3 jsonschema 4.17.3 jupyter_client 8.0.1 jupyter_core 5.1.5 jupyter-events 0.6.3 jupyter_server 2.1.0 jupyter_server_terminals 0.4.4 jupyterlab 3.5.3 jupyterlab-language-pack-zh-CN 3.5.post4 jupyterlab-pygments 0.2.2 jupyterlab_server 2.19.0 jupyterlab-widgets 3.0.5 kiwisolver 1.4.4 Markdown 3.4.1 MarkupSafe 2.1.2 matplotlib 3.6.3 matplotlib-inline 0.1.6 mistune 2.0.4 mpmath 1.3.0 multidict 6.0.4 multiprocess 0.70.15 nbclassic 0.5.1 nbclient 0.7.2 nbconvert 7.2.9 nbformat 5.7.3 nest-asyncio 1.5.6 networkx 3.2.1 ninja 1.11.1.1 notebook 6.5.2 notebook_shim 0.2.2 numpy 1.24.1 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 8.9.2.26 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu12 2.18.1 nvidia-nvjitlink-cu12 12.3.101 nvidia-nvtx-cu12 12.1.105 oauthlib 3.2.2 optimum 1.16.1 packaging 23.0 pandas 2.1.4 pandocfilters 1.5.0 parso 0.8.3 pexpect 4.8.0 pickleshare 0.7.5 Pillow 9.4.0 pip 22.3.1 platformdirs 2.6.2 pluggy 1.0.0 prometheus-client 0.16.0 prompt-toolkit 3.0.36 protobuf 3.20.3 psutil 5.9.4 ptyprocess 0.7.0 pure-eval 0.2.2 pyarrow 14.0.2 pyarrow-hotfix 0.6 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycosat 0.6.4 pycparser 2.21 Pygments 2.14.0 pyOpenSSL 22.0.0 pyparsing 3.0.9 pyrsistent 0.19.3 PySocks 1.7.1 python-dateutil 2.8.2 python-json-logger 2.0.4 pytz 2022.7.1 PyYAML 6.0 pyzmq 25.0.0 regex 2023.12.25 requests 2.28.1 requests-oauthlib 1.3.1 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rsa 4.9 ruamel.yaml 0.17.21 ruamel.yaml.clib 0.2.6 safetensors 0.4.1 scipy 1.11.4 Send2Trash 1.8.0 sentencepiece 0.1.99 setuptools 65.5.0 six 1.16.0 sniffio 1.3.0 soupsieve 2.3.2.post1 stack-data 0.6.2 sympy 1.12 tensorboard 2.11.2 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 terminado 0.17.1 tinycss2 1.2.1 tokenizers 0.15.0 tomli 2.0.1 toolz 0.12.0 torch 2.1.2 tornado 6.2 tqdm 4.64.1 traitlets 5.8.1 transformers 4.36.2 triton 2.1.0 typing_extensions 4.9.0 tzdata 2023.4 uri-template 1.2.0 urllib3 1.26.13 wcwidth 0.2.6 webcolors 1.12 webencodings 0.5.1 websocket-client 1.5.0 Werkzeug 2.2.2 wheel 0.37.1 widgetsnbextension 4.0.5 xxhash 3.4.1 yarl 1.9.4
Using
model = AirLLMLlama2("/root/autodl-tmp/ai/Yi-34B-Chat",layer_shards_saving_path="/root/autodl-tmp/ai/layerSave")
File "/root/miniconda3/lib/python3.10/site-packages/airllm/airllm.py", line 9, in init
super(AirLLMLlama2, self).init(*args, *kwargs)
File "/root/miniconda3/lib/python3.10/site-packages/airllm/airllm_base.py", line 127, in init
self.init_model()
File "/root/miniconda3/lib/python3.10/site-packages/airllm/airllm_base.py", line 187, in init_model
self.model = BetterTransformer.transform(self.model) # enable flash attention
File "/root/miniconda3/lib/python3.10/contextlib.py", line 79, in inner
return func(args, **kwds)
File "/root/miniconda3/lib/python3.10/site-packages/optimum/bettertransformer/transformation.py", line 234, in transform
raise NotImplementedError(
NotImplementedError: The model type Yi is not yet supported to be used with BetterTransformer. Feel free to open an issue at https://github.com/huggingface/optimum/issues if you would like this model type to be supported. Currently supported models are: dict_keys(['albert', 'bark', 'bart', 'bert', 'bert-generation', 'blenderbot', 'bloom', 'camembert', 'blip-2', 'clip', 'codegen', 'data2vec-text', 'deit', 'distilbert', 'electra', 'ernie', 'fsmt', 'gpt2', 'gptj', 'gpt_neo', 'gpt_neox', 'hubert', 'layoutlm', 'm2m_100', 'marian', 'markuplm', 'mbart', 'opt', 'pegasus', 'rembert', 'prophetnet', 'roberta', 'roc_bert', 'roformer', 'splinter', 'tapas', 't5', 'vilt', 'vit', 'vit_mae', 'vit_msn', 'wav2vec2', 'xlm-roberta', 'yolos']).
is_flash_attn_availableis deprecated and will be removed in v4.38. Please useis_flash_attn_2_availableinstead. Traceback (most recent call last): File "/opt/ai/test/inference_example_test.py", line 8, in请问这个怎么解决呀