sph116
commented
1 month ago 已经尝试单独下载 mit-han-lab/pile-val-backup 数据集 并且可以加载成功 但量化依旧报错
root@user-NF5288M5:/workdir/lmdeploy-main# python Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.
import os os.environ["HF_ENDPOINT"] = "https://hf-mirror.com" from datasets import load_dataset ds = load_dataset("mit-han-lab/pile-val-backup") Repo card metadata block was not found. Setting CardData to empty. Generating validation split: 214670 examples [00:37, 5766.99 examples/s]
AllentDan
Checklist
Describe the bug
量化MiniCPM-V-2_6时报错 错误信息如下 Move model.layers.0 to CPU. Move model.layers.1 to CPU. Move model.layers.2 to CPU. Move model.layers.3 to CPU. Move model.layers.4 to CPU. Move model.layers.5 to CPU. Move model.layers.6 to CPU. Move model.layers.7 to CPU. Move model.layers.8 to CPU. Move model.layers.9 to CPU. Move model.layers.10 to CPU. Move model.layers.11 to CPU. Move model.layers.12 to CPU. Move model.layers.13 to CPU. Move model.layers.14 to CPU. Move model.layers.15 to CPU. Move model.layers.16 to CPU. Move model.layers.17 to CPU. Move model.layers.18 to CPU. Move model.layers.19 to CPU. Move model.layers.20 to CPU. Move model.layers.21 to CPU. Move model.layers.22 to CPU. Move model.layers.23 to CPU. Move model.layers.24 to CPU. Move model.layers.25 to CPU. Move model.layers.26 to CPU. Move model.layers.27 to CPU. Move model.norm to GPU. Move model.rotary_emb to GPU. Move lm_head to CPU. Loading calibrate dataset ... Traceback (most recent call last): File "/opt/py3/bin/lmdeploy", line 33, in
sys.exit(load_entry_point('lmdeploy', 'console_scripts', 'lmdeploy')())
File "/workdir/lmdeploy-main/lmdeploy/cli/entrypoint.py", line 42, in run
args.run(args)
File "/workdir/lmdeploy-main/lmdeploy/cli/lite.py", line 131, in auto_awq
auto_awq(**kwargs)
File "/workdir/lmdeploy-main/lmdeploy/lite/apis/auto_awq.py", line 90, in auto_awq
vl_model, model, tokenizer, work_dir = calibrate(model,
File "/workdir/lmdeploy-main/lmdeploy/lite/apis/calibrate.py", line 235, in calibrate
calibloader, = get_calib_loaders(calib_dataset,
File "/workdir/lmdeploy-main/lmdeploy/lite/utils/calib_dataloader.py", line 323, in get_calib_loaders
return get_ptb(tokenizer, nsamples, seed, seqlen)
File "/workdir/lmdeploy-main/lmdeploy/lite/utils/calib_dataloader.py", line 64, in get_ptb
traindata = load_dataset('ptb_text_only',
File "/opt/py3/lib/python3.10/site-packages/datasets/load.py", line 2074, in load_dataset
builder_instance = load_dataset_builder(
File "/opt/py3/lib/python3.10/site-packages/datasets/load.py", line 1832, in load_dataset_builder
builder_instance: DatasetBuilder = builder_cls(
TypeError: 'NoneType' object is not callable
Reproduction
Environment
Error traceback
No response