search

abhinand5 / tamil-llama

A New Tamil Large Language Model (LLM) Based on Llama 2
GNU General Public License v3.0
236 stars 32 forks source link

ModuleNotFoundError: No module named 'flash_attn_patch' #4

Closed wenhui-huang closed 2 months ago

wenhui-huang commented 6 months ago

I use 'run_pt.sh' to pretrain llama2-7b. The following is the error message I encountered. Is it because the dependent packages are not installed?

Traceback (most recent call last): File "/mnt/d/WG-RD/lab/LLaMa2/tamil-llama/scripts/train/pretrain/run_clm_with_peft.py", line 882, in main() File "/mnt/d/WG-RD/lab/LLaMa2/tamil-llama/scripts/train/pretrain/run_clm_with_peft.py", line 450, in main from flash_attn_patch import replace_llama_attn_with_flash_attn ModuleNotFoundError: No module named 'flash_attn_patch'

pip list:

absl-py 2.0.0 accelerate 0.21.0 aiofiles 23.2.1 aiohttp 3.8.6 aiosignal 1.3.1 albumentations 1.3.1 alembic 1.12.0 altair 5.1.2 anyio 3.7.1 arrow 1.3.0 asttokens 2.4.1 async-timeout 4.0.3 attrs 23.1.0 autotrain-advanced 0.6.37 backcall 0.2.0 backports.functools-lru-cache 1.6.5 bitsandbytes 0.40.2 cachetools 5.3.1 certifi 2023.7.22 charset-normalizer 3.3.0 click 8.1.7 cmaes 0.10.0 codecarbon 2.2.3 colorlog 6.7.0 comm 0.1.4 contourpy 1.1.1 cycler 0.12.1 datasets 2.14.5 debugpy 1.6.7 decorator 5.1.1 deepspeed 0.12.4 diffusers 0.21.4 dill 0.3.7 docstring-parser 0.15 einops 0.6.1 entrypoints 0.4 evaluate 0.3.0 exceptiongroup 1.1.3 executing 2.0.1 fastapi 0.104.0 ffmpy 0.3.1 filelock 3.12.4 flash-attn 2.3.3 fonttools 4.43.1 frozenlist 1.4.0 fsspec 2023.6.0 fuzzywuzzy 0.18.0 google-auth 2.23.3 google-auth-oauthlib 1.1.0 gradio 3.41.0 gradio_client 0.5.0 greenlet 3.0.0 grpcio 1.59.0 h11 0.14.0 hjson 3.1.0 httpcore 0.18.0 httpx 0.25.0 huggingface-hub 0.17.3 idna 3.4 imageio 2.31.5 importlib-metadata 6.8.0 importlib-resources 6.1.0 inquirerpy 0.3.4 invisible-watermark 0.2.0 ipadic 1.0.0 ipykernel 6.26.0 ipython 8.16.1 jedi 0.19.1 Jinja2 3.1.2 jiwer 3.0.2 joblib 1.3.1 jsonschema 4.19.1 jsonschema-specifications 2023.7.1 jupyter-client 7.3.4 jupyter_core 5.5.0 kiwisolver 1.4.5 lazy_loader 0.3 loguru 0.7.0 Mako 1.2.4 Markdown 3.5 markdown-it-py 3.0.0 MarkupSafe 2.1.3 matplotlib 3.8.0 matplotlib-inline 0.1.6 mdurl 0.1.2 mpmath 1.3.0 multidict 6.0.4 multiprocess 0.70.15 nest-asyncio 1.5.8 networkx 3.2 ninja 1.11.1.1 numpy 1.26.1 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 8.9.2.26 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu12 2.18.1 nvidia-nvjitlink-cu12 12.3.52 nvidia-nvtx-cu12 12.1.105 oauthlib 3.2.2 opencv-python 4.8.1.78 opencv-python-headless 4.8.1.78 optuna 3.3.0 orjson 3.9.9 packaging 23.1 pandas 2.1.1 parso 0.8.3 peft 0.4.0 pexpect 4.8.0 pfzy 0.3.4 pickleshare 0.7.5 Pillow 10.0.0 pip 23.3.1 platformdirs 3.11.0 prompt-toolkit 3.0.39 protobuf 4.23.4 psutil 5.9.6 ptyprocess 0.7.0 pure-eval 0.2.2 py-cpuinfo 9.0.0 pyarrow 13.0.0 pyasn1 0.5.0 pyasn1-modules 0.3.0 pydantic 1.10.11 pydub 0.25.1 Pygments 2.16.1 pynvml 11.5.0 pyparsing 3.1.1 python-dateutil 2.8.2 python-multipart 0.0.6 pytz 2023.3.post1 PyWavelets 1.4.1 PyYAML 6.0.1 pyzmq 25.1.0 qudida 0.0.4 rapidfuzz 2.13.7 referencing 0.30.2 regex 2023.10.3 requests 2.31.0 requests-oauthlib 1.3.1 responses 0.18.0 rich 13.6.0 rpds-py 0.10.6 rsa 4.9 sacremoses 0.0.53 safetensors 0.4.0 scikit-image 0.22.0 scikit-learn 1.3.0 scipy 1.11.3 semantic-version 2.10.0 sentencepiece 0.1.99 setuptools 68.0.0 shtab 1.6.4 six 1.16.0 sniffio 1.3.0 SQLAlchemy 2.0.23 stack-data 0.6.3 starlette 0.27.0 sympy 1.12 tensorboard 2.15.0 tensorboard-data-server 0.7.1 threadpoolctl 3.2.0 tifffile 2023.9.26 tiktoken 0.5.2 tokenizers 0.13.3 toolz 0.12.0 torch 2.1.0 tornado 6.1 tqdm 4.65.0 traitlets 5.12.0 transformers 4.31.0 triton 2.1.0 trl 0.4.7 types-python-dateutil 2.8.19.14 typing_extensions 4.8.0 tyro 0.5.10 tzdata 2023.3 urllib3 2.0.7 uvicorn 0.23.2 wcwidth 0.2.8 websockets 11.0.3 Werkzeug 2.3.6 wheel 0.41.2 xformers 0.0.22.post4 xgboost 1.7.6 xxhash 3.4.1 yarl 1.9.2 zipp 3.17.0

abhinand5 commented 6 months ago

Hi @wenhui-huang , the issue is because of the patch script not being pushed to the repo. You can pull the changes and try again.

Note: Now with the recent updates to the transformers package, support for flash attention 2 comes inbuilt. I will be rolling out an upgrade in the coming days.