TheHonestBob
commented
1 year ago Is there an existing issue for this?
- [x] I have searched the existing issues
Current Behavior
运行代码到：peft_model = get_peft_model(model, peft_config) 时报错 ValueError: Target module QuantizedLinear() is not supported. Currently, only
torch.nn.LinearandConv1Dare supported.Expected Behavior
No response
Steps To Reproduce
运行代码到：peft_model = get_peft_model(model, peft_config) 时报错 print("** 定义模型 ***") from peft import get_peft_model, AdaLoraConfig, TaskType
训练时节约GPU占用 model.config.use_cache = True model.supports_gradient_checkpointing = True # model.gradient_checkpointing_enable() model.enable_input_require_grads()
peft_config = AdaLoraConfig( task_type=TaskType.CAUSAL_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1, target_modules=["query", "value"] )
peft_model = get_peft_model(model, peft_config)
Environment
# Name Version Build Channel _libgcc_mutex 0.1 conda_forge https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge _openmp_mutex 4.5 2_kmp_llvm https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge accelerate 0.20.3 pypi_0 pypi aiofiles 23.1.0 pypi_0 pypi aiohttp 3.8.4 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi altair 5.0.1 pypi_0 pypi anyio 3.7.0 pypi_0 pypi aom 3.5.0 h27087fc_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge async-timeout 4.0.2 pypi_0 pypi attrs 23.1.0 pypi_0 pypi blas 1.0 mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main blinker 1.6.2 pypi_0 pypi brotlipy 0.7.0 py311h9bf148f_1002 pytorch-nightly bzip2 1.0.8 h7f98852_4 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge ca-certificates 2023.5.7 hbcca054_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge cachetools 5.3.1 pypi_0 pypi cairo 1.16.0 hbbf8b49_1016 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge certifi 2023.5.7 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge cffi 1.15.1 py311h9bf148f_3 pytorch-nightly charset-normalizer 2.1.1 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge click 8.1.3 pypi_0 pypi contourpy 1.1.0 pypi_0 pypi cpm-kernels 1.0.11 pypi_0 pypi cryptography 38.0.4 py311h46ebde7_0 pytorch-nightly cuda-cudart 12.1.105 0 nvidia cuda-cupti 12.1.105 0 nvidia cuda-libraries 12.1.0 0 nvidia cuda-nvrtc 12.1.105 0 nvidia cuda-nvtx 12.1.105 0 nvidia cuda-opencl 12.1.105 0 nvidia cuda-runtime 12.1.0 0 nvidia cycler 0.11.0 pypi_0 pypi datasets 2.12.0 pypi_0 pypi dav1d 1.2.1 hd590300_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge decorator 5.1.1 pypi_0 pypi deepspeed 0.9.3 pypi_0 pypi dill 0.3.6 pypi_0 pypi expat 2.5.0 hcb278e6_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge fastapi 0.98.0 pypi_0 pypi ffmpeg 6.0.0 gpl_h17d8df4_102 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge ffmpy 0.3.0 pypi_0 pypi filelock 3.9.0 py311_0 pytorch-nightly font-ttf-dejavu-sans-mono 2.37 hab24e00_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge font-ttf-inconsolata 3.000 h77eed37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge font-ttf-ubuntu 0.83 hab24e00_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge fontconfig 2.14.2 h14ed4e7_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge fonts-conda-ecosystem 1 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge fonts-conda-forge 1 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge fonttools 4.40.0 pypi_0 pypi freetype 2.12.1 hca18f0e_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge fribidi 1.0.10 h36c2ea0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge frozenlist 1.3.3 pypi_0 pypi fsspec 2023.5.0 pypi_0 pypi gettext 0.21.1 h27087fc_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge giflib 5.2.1 h0b41bf4_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge git-lfs 1.6 pypi_0 pypi gitdb 4.0.10 pypi_0 pypi gitpython 3.1.31 pypi_0 pypi gmp 6.2.1 h58526e2_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge gmpy2 2.1.2 py311h6a5fa03_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge gnutls 3.7.8 hf3e180e_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge gradio 3.35.2 pypi_0 pypi gradio-client 0.2.7 pypi_0 pypi graphite2 1.3.13 h58526e2_1001 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge h11 0.14.0 pypi_0 pypi harfbuzz 7.3.0 hdb3a94d_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge hjson 3.1.0 pypi_0 pypi httpcore 0.17.2 pypi_0 pypi httpx 0.24.1 pypi_0 pypi huggingface-hub 0.14.1 pypi_0 pypi icu 72.1 hcb278e6_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge idna 3.4 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge importlib-metadata 6.7.0 pypi_0 pypi jieba 0.42.1 pypi_0 pypi jinja2 3.1.2 pypi_0 pypi joblib 1.3.1 pypi_0 pypi jpeg 9e h0b41bf4_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge jsonschema 4.17.3 pypi_0 pypi kiwisolver 1.4.4 pypi_0 pypi lame 3.100 h166bdaf_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge latex2mathml 3.76.0 pypi_0 pypi lcms2 2.15 hfd0df8a_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge ld_impl_linux-64 2.40 h41732ed_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge lerc 4.0.0 h27087fc_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libass 0.17.1 hc9aadba_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libcublas 12.1.0.26 0 nvidia libcufft 11.0.2.4 0 nvidia libcufile 1.6.1.9 0 nvidia libcurand 10.3.2.106 0 nvidia libcusolver 11.4.4.55 0 nvidia libcusparse 12.0.2.55 0 nvidia libdeflate 1.17 h0b41bf4_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libdrm 2.4.114 h166bdaf_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libexpat 2.5.0 hcb278e6_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libffi 3.4.2 h7f98852_5 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libgcc-ng 13.1.0 he5830b7_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libglib 2.76.3 hebfc3b9_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libhwloc 2.9.1 nocuda_h7313eea_6 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libiconv 1.17 h166bdaf_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libidn2 2.3.4 h166bdaf_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libnpp 12.0.2.50 0 nvidia libnsl 2.0.0 h7f98852_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libnvjitlink 12.1.105 0 nvidia libnvjpeg 12.1.0.39 0 nvidia libopus 1.3.1 h7f98852_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libpciaccess 0.17 h166bdaf_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libpng 1.6.39 h753d276_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libsqlite 3.42.0 h2797004_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libstdcxx-ng 13.1.0 hfd8a6a1_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libtasn1 4.19.0 h166bdaf_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libtiff 4.5.0 h6adf6a1_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libunistring 0.9.10 h7f98852_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libuuid 2.38.1 h0b41bf4_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libva 2.18.0 h0b41bf4_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libvpx 1.13.0 hcb278e6_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libwebp 1.2.4 h1daa5a0_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libwebp-base 1.2.4 h166bdaf_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libxcb 1.15 h0b41bf4_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libxml2 2.11.4 h0d562d8_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libzlib 1.2.13 h166bdaf_4 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge linkify-it-py 2.0.2 pypi_0 pypi llvm-openmp 16.0.5 h4dfa4b3_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge markdown 3.4.3 pypi_0 pypi markdown-it-py 2.2.0 pypi_0 pypi markupsafe 2.1.3 pypi_0 pypi matplotlib 3.7.1 pypi_0 pypi mdit-py-plugins 0.3.3 pypi_0 pypi mdtex2html 1.2.0 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi mkl 2021.4.0 h8d4b97c_729 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge mkl-service 2.4.0 py311h9bf148f_0 pytorch-nightly mkl_fft 1.3.1 py311hc796f24_0 pytorch-nightly mkl_random 1.2.2 py311hbba84a0_0 pytorch-nightly mpc 1.3.1 hfe3b2da_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge mpfr 4.2.0 hb012696_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge mpmath 1.2.1 py311_0 pytorch-nightly multidict 6.0.4 pypi_0 pypi multiprocess 0.70.14 pypi_0 pypi ncurses 6.4 hcb278e6_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge nettle 3.8.1 hc379101_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge networkx 3.1 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge ninja 1.11.1 pypi_0 pypi nltk 3.8.1 pypi_0 pypi numpy 1.24.3 py311hc206e33_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main numpy-base 1.24.3 py311hfd5febd_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main openh264 2.3.1 hcb278e6_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge openssl 3.1.1 hd590300_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge orjson 3.9.1 pypi_0 pypi p11-kit 0.24.1 hc5aa10d_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge packaging 23.1 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge pandas 2.0.2 pypi_0 pypi pcre2 10.40 hc3806b6_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge peft 0.3.0 pypi_0 pypi pillow 9.3.0 py311h3fd9d12_2 pytorch-nightly pip 23.1.2 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge pixman 0.40.0 h36c2ea0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge protobuf 3.20.3 pypi_0 pypi psutil 5.9.5 py311h2582759_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge pthread-stubs 0.4 h36c2ea0_1001 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge py-cpuinfo 9.0.0 pypi_0 pypi pyarrow 12.0.0 pypi_0 pypi pycparser 2.21 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge pydantic 1.10.8 pypi_0 pypi pydeck 0.8.1b0 pypi_0 pypi pydub 0.25.1 pypi_0 pypi pygments 2.15.1 pypi_0 pypi pympler 1.0.1 pypi_0 pypi pyopenssl 23.2.0 pyhd8ed1ab_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge pyparsing 3.1.0 pypi_0 pypi pyrsistent 0.19.3 pypi_0 pypi pysocks 1.7.1 py311_0 pytorch-nightly python 3.11.3 h2755cc3_0_cpython https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge python-dateutil 2.8.2 pypi_0 pypi python-multipart 0.0.6 pypi_0 pypi python_abi 3.11 3_cp311 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge pytorch 2.1.0.dev20230607 py3.11_cuda12.1_cudnn8.8.1_0 pytorch-nightly pytorch-cuda 12.1 ha16c6d3_5 pytorch-nightly pytorch-mutex 1.0 cuda pytorch-nightly pytz 2023.3 pypi_0 pypi pytz-deprecation-shim 0.1.0.post0 pypi_0 pypi pyyaml 6.0 py311hd4cff14_5 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge readline 8.2 h8228510_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge regex 2023.5.5 pypi_0 pypi requests 2.28.1 py311_0 pytorch-nightly responses 0.18.0 pypi_0 pypi rich 13.4.2 pypi_0 pypi rouge-chinese 1.0.3 pypi_0 pypi safetensors 0.3.1 pypi_0 pypi semantic-version 2.10.0 pypi_0 pypi sentencepiece 0.1.99 pypi_0 pypi setuptools 67.7.2 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge six 1.16.0 pyh6c4a22f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge smmap 5.0.0 pypi_0 pypi sniffio 1.3.0 pypi_0 pypi starlette 0.27.0 pypi_0 pypi streamlit 1.24.0 pypi_0 pypi streamlit-chat 0.1.1 pypi_0 pypi svt-av1 1.5.0 h59595ed_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge sympy 1.12 pypyh9d50eac_103 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge tbb 2021.9.0 hf52228f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge tenacity 8.2.2 pypi_0 pypi tk 8.6.12 h27826a3_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge tokenizers 0.13.3 pypi_0 pypi toml 0.10.2 pypi_0 pypi toolz 0.12.0 pypi_0 pypi torchaudio 2.1.0.dev20230608 py311_cu121 pytorch-nightly torchkeras 3.9.0 pypi_0 pypi torchtriton 2.1.0+9820899b38 py311 pytorch-nightly torchvision 0.16.0.dev20230607 py311_cu121 pytorch-nightly tornado 6.3.2 pypi_0 pypi tqdm 4.65.0 pypi_0 pypi transformers 4.29.2 pypi_0 pypi typing_extensions 4.6.3 pyha770c72_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge tzdata 2023.3 pypi_0 pypi tzlocal 4.3.1 pypi_0 pypi uc-micro-py 1.0.2 pypi_0 pypi urllib3 1.26.14 py311_0 pytorch-nightly uvicorn 0.22.0 pypi_0 pypi validators 0.20.0 pypi_0 pypi watchdog 3.0.0 pypi_0 pypi websockets 11.0.3 pypi_0 pypi wheel 0.40.0 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge x264 1!164.3095 h166bdaf_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge x265 3.5 h924138e_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-fixesproto 5.0 h7f98852_1002 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-kbproto 1.0.7 h7f98852_1002 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-libice 1.1.1 hd590300_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-libsm 1.2.4 h7391055_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-libx11 1.8.5 h8ee46fc_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-libxau 1.0.11 hd590300_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-libxext 1.3.4 h0b41bf4_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-libxfixes 5.0.3 h7f98852_1004 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-libxrender 0.9.10 h7f98852_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-renderproto 0.11.1 h7f98852_1002 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-xextproto 7.3.0 h0b41bf4_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xorg-xproto 7.0.31 h7f98852_1007 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge xxhash 3.2.0 pypi_0 pypi xz 5.2.6 h166bdaf_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge yaml 0.2.5 h7f98852_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge yarl 1.9.2 pypi_0 pypi zipp 3.15.0 pypi_0 pypi zlib 1.2.13 h166bdaf_4 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge zstd 1.5.2 h3eb15da_6 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forgeAnything else?
No response
我和你在做一样的工作，我也是遇到这个问题，不知道是不是int4训练应该使用Qlora，现在还没有时间去尝试
yushuaizhao
Grason-Lu
hubery339
yangkai1129
Is there an existing issue for this?
Current Behavior
运行代码到：peft_model = get_peft_model(model, peft_config) 时报错 ValueError: Target module QuantizedLinear() is not supported. Currently, only
torch.nn.LinearandConv1Dare supported.Expected Behavior
No response
Steps To Reproduce
运行代码到：peft_model = get_peft_model(model, peft_config) 时报错 print("** 定义模型 ***") from peft import get_peft_model, AdaLoraConfig, TaskType
训练时节约GPU占用
model.config.use_cache = True model.supports_gradient_checkpointing = True # model.gradient_checkpointing_enable() model.enable_input_require_grads()
peft_config = AdaLoraConfig( task_type=TaskType.CAUSAL_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1, target_modules=["query", "value"] )
peft_model = get_peft_model(model, peft_config)
Environment
Anything else?
No response