search

OpenBMB / MiniCPM-V

MiniCPM-Llama3-V 2.5: A GPT-4V Level Multimodal LLM on Your Phone
Apache License 2.0
7.98k stars 558 forks source link

我的M3芯片本地运行MiniCPM-Llama3-V-2_5-int4得到了报错Using `bitsandbytes` 8-bit quantization requires Accelerate #215

Open myBigbug opened 1 month ago

myBigbug commented 1 month ago

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

当前行为 | Current Behavior

我已经在conda环境中安装了所需要的依赖，在本地mac上执行PYTORCH_ENABLE_MPS_FALLBACK=1 python xxx.py命令的时候，得到了报错。

期望行为 | Expected Behavior

得到输出

复现方法 | Steps To Reproduce

执行PYTORCH_ENABLE_MPS_FALLBACK=1 python xxx.py

运行环境 | Environment

- OS: Macos M3 sonoma14.3
- Python: 3.10
- Transformers:4.40.0
- PyTorch: 2.1.2
- CUDA : None，使用的MPS

其他pip安装的环境
Package                   Version
------------------------- ------------
accelerate                0.30.1
addict                    2.4.0
aiofiles                  23.2.1
altair                    5.3.0
annotated-types           0.7.0
anyio                     4.4.0
attrs                     23.2.0
bitsandbytes              0.42.0
blis                      0.7.11
Brotli                    1.1.0
catalogue                 2.0.10
certifi                   2024.2.2
charset-normalizer        3.3.2
click                     8.1.7
cloudpathlib              0.16.0
colorama                  0.4.6
confection                0.1.5
contourpy                 1.2.1
cycler                    0.12.1
cymem                     2.0.8
editdistance              0.6.2
einops                    0.7.0
et-xmlfile                1.1.0
exceptiongroup            1.2.1
fairscale                 0.4.0
fastapi                   0.110.3
ffmpy                     0.3.2
filelock                  3.14.0
fonttools                 4.53.0
fsspec                    2024.5.0
gmpy2                     2.1.5
gradio                    4.26.0
gradio_client             0.15.1
h11                       0.14.0
httpcore                  1.0.5
httpx                     0.27.0
huggingface_hub           0.23.2
idna                      3.7
importlib_resources       6.4.0
Jinja2                    3.1.4
joblib                    1.4.2
jsonlines                 4.0.0
jsonschema                4.22.0
jsonschema-specifications 2023.12.1
kiwisolver                1.4.5
langcodes                 3.4.0
language_data             1.2.0
lxml                      5.2.2
marisa-trie               1.1.1
markdown-it-py            3.0.0
markdown2                 2.4.10
MarkupSafe                2.1.5
matplotlib                3.7.4
mdurl                     0.1.2
more-itertools            10.1.0
mpmath                    1.3.0
murmurhash                1.0.10
networkx                  3.3
nltk                      3.8.1
numpy                     1.24.4
opencv-python-headless    4.5.5.64
openpyxl                  3.1.2
orjson                    3.10.3
packaging                 23.2
pandas                    2.2.2
Pillow                    10.1.0
pip                       24.0
portalocker               2.8.2
preshed                   3.0.9
protobuf                  4.25.0
psutil                    5.9.8
pydantic                  2.7.3
pydantic_core             2.18.4
pydub                     0.25.1
Pygments                  2.18.0
pyparsing                 3.1.2
PySocks                   1.7.1
python-dateutil           2.9.0.post0
python-multipart          0.0.9
pytz                      2024.1
PyYAML                    6.0.1
referencing               0.35.1
regex                     2024.5.15
requests                  2.32.3
rich                      13.7.1
rpds-py                   0.18.1
ruff                      0.4.7
sacrebleu                 2.3.2
safetensors               0.4.3
scipy                     1.13.1
seaborn                   0.13.0
semantic-version          2.10.0
sentencepiece             0.1.99
setuptools                70.0.0
shellingham               1.5.4
shortuuid                 1.0.11
six                       1.16.0
smart-open                6.4.0
sniffio                   1.3.1
socksio                   1.0.0
spacy                     3.7.2
spacy-legacy              3.0.12
spacy-loggers             1.0.5
srsly                     2.4.8
starlette                 0.37.2
sympy                     1.12.1
tabulate                  0.9.0
thinc                     8.2.3
timm                      0.9.10
tokenizers                0.19.1
tomlkit                   0.12.0
toolz                     0.12.1
torch                     2.1.2
torchvision               0.16.2
tqdm                      4.66.1
transformers              4.40.0
typer                     0.9.4
typing_extensions         4.12.1
tzdata                    2024.1
urllib3                   2.2.1
uvicorn                   0.24.0.post1
wasabi                    1.1.3
weasel                    0.3.4
websockets                11.0.3
wheel                     0.43.0

备注 | Anything else?

日志：


lib'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
Unused kwargs: ['_load_in_4bit', '_load_in_8bit', 'quant_method']. These kwargs are not used in <class 'transformers.utils.quantization_config.BitsAndBytesConfig'>.
Traceback (most recent call last):
  File "/Users/xxx/miniCPM/local_main.py", line 7, in <module>
    model = AutoModel.from_pretrained('/Users/gaobo60/aiModel/MiniCPM-Llama3-V-2_5-int4', trust_remote_code=True)
  File "/Users/xxx/miniCPM/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
    return model_class.from_pretrained(
  File "/Users/xxxxx/miniCPM/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3165, in from_pretrained
    hf_quantizer.validate_environment(
  File "/Users/xxxx/miniCPM/lib/python3.10/site-packages/transformers/quantizers/quantizer_bnb_4bit.py", line 62, in validate_environment
    raise ImportError(
ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`
myBigbug commented 1 month ago

我已经使用命令 pip install -i https://pypi.org/simple/ bitsandbytes ，之后仍然报错

iceflame89 commented 1 month ago

由于bitsandbytes (see issse)现在还不支持MPS，int4模型暂时无法在mac运行。

myBigbug commented 1 month ago

由于bitsandbytes (see issse)现在还不支持MPS，int4模型暂时无法在mac运行。

感谢解答，等支持后再Close该issue

annatongtong commented 1 month ago

由于bitsandbytes (see issse)现在还不支持MPS，int4模型暂时无法在mac运行。

请问目前支持win10吗？我setup了requirements和bitsandbytes，都提示 raise ImportError( ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes

iceflame89 commented 4 weeks ago

由于bitsandbytes (see issse)现在还不支持MPS，int4模型暂时无法在mac运行。

请问目前支持win10吗？我setup了requirements和bitsandbytes，都提示 raise ImportError( ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes

bitsandbytes 目前仅支持cuda设备

annatongtong commented 3 weeks ago

非常感谢

在 2024-06-17 20:37:17，"Hongji Zhu" @.***> 写道：

由于bitsandbytes (see issse)现在还不支持MPS，int4模型暂时无法在mac运行。

请问目前支持win10吗？我setup了requirements和bitsandbytes，都提示 raise ImportError( ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes

bitsandbytes 目前仅支持cuda设备

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

vizshrc commented 1 week ago

+1

sunny6206 commented 13 hours ago

I'm trying to fine tune "openbmb/MiniCPM-Llama3-V-2_5-int4" with custom dataset. but getting error "[2024-07-16 11:22:47,459] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] async_io: please install the libaio-dev package with apt [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH [WARNING] sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1 [WARNING] using untested triton version (2.1.0), only 1.0.0 is known to be compatible /mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/transformers/training_args.py:1474: FutureWarning: evaluation_strategy is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use eval_strategy instead warnings.warn( [2024-07-16 11:23:00,796] [INFO] [comm.py:637:init_distributed] cdb=None [2024-07-16 11:23:00,796] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl Unused kwargs: ['_load_in_4bit', '_load_in_8bit', 'quant_method']. These kwargs are not used in <class 'transformers.utils.quantization_config.BitsAndBytesConfig'>. low_cpu_mem_usage was None, now set to True since model is quantized. Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:35<00:00, 17.78s/it] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Currently using LoRA for fine-tuning the MiniCPM-V model. Traceback (most recent call last): File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/MiniCPM-V-main/finetune/finetune.py", line 328, in train() File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/MiniCPM-V-main/finetune/finetune.py", line 274, in train model.base_model.vpm.requiresgrad(True) File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2440, in requiresgrad p.requiresgrad(requires_grad) RuntimeError: only Tensors of floating point dtype can require gradients [2024-07-16 11:24:03,027] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 129003) of binary: /mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/bin/python3 Traceback (most recent call last): File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/bin/torchrun", line 8, in sys.exit(main()) File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper return f(*args, **kwargs) File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/run.py", line 806, in main run(args) File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run elastic_launch( File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in call return launch_agent(self._config, self._entrypoint, list(args)) File "/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

finetune.py FAILED

Failures:

------------------------------------------------------------ Root Cause (first observed failure): [0]: time : 2024-07-16_11:24:02 host : AlticeLab. rank : 0 (local_rank: 0) exitcode : 1 (pid: 129003) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html" the code is "#!/bin/bash GPUS_PER_NODE=1 NNODES=1 NODE_RANK=0 MASTER_ADDR=localhost MASTER_PORT=6001 MODEL="openbmb/MiniCPM-Llama3-V-2_5-int4" # or openbmb/MiniCPM-V-2 # ATTENTION: specify the path to your training data, which should be a json file consisting of a list of conversations. # See the section for finetuning in README for more information. DATA="/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/MiniCPM-V-main/finetune/vl_finetune_data.json" EVAL_DATA="/mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/MiniCPM-V-main/finetune/finetune-evaluate_data.json" LLM_TYPE="minicpm" # if use openbmb/MiniCPM-V-2, please set LLM_TYPE=minicpm DISTRIBUTED_ARGS=" --nproc_per_node $GPUS_PER_NODE \ --nnodes $NNODES \ --node_rank $NODE_RANK \ --master_addr $MASTER_ADDR \ --master_port $MASTER_PORT " /mnt/c/Users/akhil/OneDrive/Desktop/WITBE_INTA/llm/linux-env/bin/torchrun $DISTRIBUTED_ARGS finetune.py \ --model_name_or_path $MODEL \ --llm_type $LLM_TYPE \ --data_path $DATA \ --eval_data_path $EVAL_DATA \ --remove_unused_columns false \ --label_names "labels" \ --prediction_loss_only false \ --bf16 false \ --bf16_full_eval false \ --fp16 true \ --fp16_full_eval true \ --do_train \ --do_eval \ --tune_vision true \ --tune_llm false \ --use_lora true \ --lora_target_modules "llm\..*layers\.\d+\.self_attn\.(q_proj|k_proj)" \ --model_max_length 2048 \ --max_slice_nums 9 \ --max_steps 10000 \ --eval_steps 1000 \ --output_dir output/output_minicpmv2_lora \ --logging_dir output/output_minicpmv2_lora \ --logging_strategy "steps" \ --per_device_train_batch_size 2 \ --per_device_eval_batch_size 1 \ --gradient_accumulation_steps 1 \ --evaluation_strategy "steps" \ --save_strategy "steps" \ --save_steps 1000 \ --save_total_limit 10 \ --learning_rate 1e-6 \ --weight_decay 0.1 \ --adam_beta2 0.95 \ --warmup_ratio 0.01 \ --lr_scheduler_type "cosine" \ --logging_steps 1 \ --gradient_checkpointing true \ --deepspeed ds_config_zero2.json \ --report_to "tensorboard" # wandb "
sunny6206 commented 13 hours ago

Also I have set the correct path for json dataset