search

modelscope / ms-swift

Use PEFT or Full-parameter to finetune 350+ LLMs or 90+ MLLMs. (Qwen2.5, GLM4v, Internlm2.5, Yi, Llama3.1, Llava-Video, Internvl2, MiniCPM-V-2.6, Deepseek, Baichuan2, Gemma2, Phi3-Vision, ...)
https://swift.readthedocs.io/zh-cn/latest/Instruction/index.html
Apache License 2.0
3.48k stars 298 forks source link

qwen-32B自我认知训练完 int4量化时报错 assert model_name is not None and model_author is not None #907

Closed xudongLi-Alex closed 4 months ago

xudongLi-Alex commented 4 months ago

问题描述： 可以正常export模型并推理，但是量化的时候报错，应该是数据集的原因

命令：

CUDA_VISIBLE_DEVICES=0,1 swift export \ --ckpt_dir "/home/user/sdb1/sft-output/qwen1half-32b-chat/v4-20240510-064821/checkpoint-50/" \ --merge_lora true \ --quant_bits 4 \ --load_dataset_config true --quant_method awq

报错内容： [INFO:swift] Qwen2AWQForCausalLM: 32512.2181M Params (32512.2181M Trainable [100.0000%]), 536.8750M Buffers. [INFO:swift] system: You are a helpful assistant. [INFO:swift] Quantization dataset: ['alpaca-zh#500', 'alpaca-en#500', 'self-cognition#500'] [INFO:swift] Start quantizing the model... [INFO:swift] Downloading the dataset from ModelScope, dataset_id: AI-ModelScope/alpaca-gpt4-data-zh [INFO:modelscope] dataset_type: 4 [INFO:modelscope] Context manager of ms-dataset exited. 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 5084.87it/s] [INFO:swift] Downloading the dataset from ModelScope, dataset_id: AI-ModelScope/alpaca-gpt4-data-en [INFO:modelscope] dataset_type: 4 [INFO:modelscope] Context manager of ms-dataset exited. 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1997.53it/s] Traceback (most recent call last): File "/home/user/sdb1/swift-main/swift/cli/export.py", line 5, in export_main() File "/home/user/sdb1/swift-main/swift/utils/run_utils.py", line 27, in x_main result = llm_x(args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/sdb1/swift-main/swift/llm/export.py", line 106, in llm_export awq_model_quantize(model, template.tokenizer) File "/home/user/sdb1/swift-main/swift/llm/export.py", line 65, in awq_model_quantize awq_model.quantize(tokenizer, quant_config=quant_config) File "/home/user/miniconda3/envs/swift/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/miniconda3/envs/swift/lib/python3.11/site-packages/awq/models/base.py", line 170, in quantize self.quantizer = AwqQuantizer( ^^^^^^^^^^^^^ File "/home/user/miniconda3/envs/swift/lib/python3.11/site-packages/awq/quantize/quantizer.py", line 61, in init self.modules, self.module_kwargs, self.inps = self.init_quant() ^^^^^^^^^^^^^^^^^ File "/home/user/miniconda3/envs/swift/lib/python3.11/site-packages/awq/quantize/quantizer.py", line 440, in init_quant samples = get_calib_dataset( ^^^^^^^^^^^^^^^^^^ File "/home/user/sdb1/swift-main/swift/llm/export.py", line 25, in _get_dataset dataset = get_dataset(data)[0] ^^^^^^^^^^^^^^^^^ File "/home/user/sdb1/swift-main/swift/llm/utils/dataset.py", line 1261, in get_dataset assert model_name is not None and model_author is not None ^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError

TZJ12 commented 4 months ago

请问你这个问题解决了嘛，我也出了同样的错误

xudongLi-Alex commented 4 months ago

请问你这个问题解决了嘛，我也出了同样的错误

还没有正在找原因