对模型量化4bit时报错ConnectionError: Couldn't reach 'ptb_text_only' on the Hub (ConnectionError)

sxk000 commented 3 months ago

Checklist

[x] 1. I have searched related issues but cannot get the expected help.
[x] 2. The bug has not been fixed in the latest version.
[x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

首先，感谢上海人工智能实验室及其成员对书生模型、代码框架、技术经验的分享！

最近在用lmdeploy量化internlm2-chat-20b为4bit时报错，该如何解决呢？

谢谢！

Reproduction

量化运行命令：lmdeploy lite auto_awq internlm2-chat-20b --work-dir internlm2-chat-20b-4bit

Environment

环境：
cuda12.2
python3.12
lmdeploy0.5.1

Error traceback

报错如下：
Move model.layers.46 to CPU.
Move model.layers.47 to CPU.
Move model.norm to GPU.
Move output to CPU.
Loading calibrate dataset ...
Traceback (most recent call last):
  File "/root/miniconda3/envs/p312lmdeploy/bin/lmdeploy", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/lmdeploy/cli/entrypoint.py", line 36, in run
    args.run(args)
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/lmdeploy/cli/lite.py", line 139, in auto_awq
    auto_awq(**kwargs)
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/lmdeploy/lite/apis/auto_awq.py", line 103, in auto_awq
    vl_model, model, tokenizer, work_dir = calibrate(model,
                                           ^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/lmdeploy/lite/apis/calibrate.py", line 209, in calibrate
    calib_loader, _ = get_calib_loaders(calib_dataset,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/lmdeploy/lite/utils/calib_dataloader.py", line 302, in get_calib_loaders
    return get_ptb(tokenizer, nsamples, seed, seqlen)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/lmdeploy/lite/utils/calib_dataloader.py", line 58, in get_ptb
    traindata = load_dataset('ptb_text_only', 'penn_treebank', split='train')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/datasets/load.py", line 2594, in load_dataset
    builder_instance = load_dataset_builder(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/datasets/load.py", line 2266, in load_dataset_builder
    dataset_module = dataset_module_factory(
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/datasets/load.py", line 1914, in dataset_module_factory
    raise e1 from None
  File "/root/miniconda3/envs/p312lmdeploy/lib/python3.12/site-packages/datasets/load.py", line 1845, in dataset_module_factory
    raise ConnectionError(f"Couldn't reach '{path}' on the Hub ({e.__class__.__name__})") from e
ConnectionError: Couldn't reach 'ptb_text_only' on the Hub (ConnectionError)

lvhan028 commented 3 months ago

正如日志所示，连不上huggingface，导致找不到 ptb_text_only。或许可以看看能不能想办法连上 huggingface 呢？

sxk000 commented 3 months ago

正如日志所示，连不上huggingface，导致找不到 ptb_text_only。或许可以看看能不能想办法连上 huggingface 呢？

嗯，不是所有的环境都是可以连上 huggingface的，如果把链接 huggingface写死到框架里面，势必会导致很多不能使用的情况，不方便使用。

所以对于不能链接到 huggingface上的情况，能不能做优化呢？

AllentDan commented 3 months ago

Possible solutions: https://github.com/InternLM/lmdeploy/issues/1142#issuecomment-2072588598 https://github.com/InternLM/lmdeploy/issues/1865

sxk000 commented 3 months ago

Possible solutions: #1142 (comment) #1865

谢谢解答，根据上面提到的方法，问题解决了！

NB-Group commented 3 months ago

我遇到了和@sxk00一样的问题。但是我的网络环境是可以访问hugging face的。但是也会报错： ConnectionError: Couldn't reach 'ptb_text_only' on the Hub (ProxyError)

另外报错的位置和信息有略微不同：

  File "E:\software\Anaconda3\envs\MOSS_2.0\lib\site-packages\datasets\load.py", line 1468, in dataset_module_factory
    raise ConnectionError(f"Couldn't reach '{path}' on the Hub ({type(e).__name__})")
ConnectionError: Couldn't reach 'ptb_text_only' on the Hub (ProxyError)

sxk000 commented 3 months ago

Possible solutions: #1142 (comment) #1865

@NB-Group 属于同一类问题，可以根据上面提到的方法解决。

NB-Group commented 3 months ago

我已经解决了，问题出在urllib库对vpn代理地址的获取上。在环境变量里面设置一下就行了

InternLM / lmdeploy