Repo for BenTsao [original name: HuaTuo (华驼)], Instruction-tuning Large Language Models with Chinese Medical Knowledge. 本草（原名：华驼）模型仓库，基于中文医学知识的大语言模型指令微调
Apache License 2.0
4.31k
stars
422
forks
source link
NotImplementedError: Cannot copy out of meta tensor; no data! #75
sjtuzhaoxh
commented
10 months ago
│ │ │ 403 │ │ │ │ device_map = infer_auto_device_map( │ │ 404 │ │ │ │ │ self, max_memory=max_memory, no_split_module_classes=no_split_module │ │ 405 │ │ │ │ ) │ │ ❱ 406 │ │ │ dispatch_model( │ │ 407 │ │ │ │ self, │ │ 408 │ │ │ │ device_map=device_map, │ │ 409 │ │ │ │ offload_dir=offload_dir, │ │ │ │ C:\Users\zhaoxianghui\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\big_m │ │ odeling.py:355 in dispatch_model │ │ │ │ 352 │ │ and (not os.path.isdir(offload_dir) or not os.path.isfile(os.path.join(offload_d │ │ 353 │ ): │ │ 354 │ │ disk_state_dict = extract_submodules_state_dict(model.state_dict(), disk_modules │ │ ❱ 355 │ │ offload_state_dict(offload_dir, disk_state_dict) │ │ 356 │ │ │ 357 │ execution_device = { │ │ 358 │ │ name: main_device if device in ["cpu", "disk"] else device for name, device in d │ │ │ │ C:\Users\zhaoxianghui\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\utils │ │ \offload.py:103 in offload_state_dict │ │ │ │ 100 │ os.makedirs(save_dir, exist_ok=True) │ │ 101 │ index = {} │ │ 102 │ for name, parameter in state_dict.items(): │ │ │ │ 34 │ │ # Need to reinterpret the underlined data as int16 since NumPy does not handle b │ │ 35 │ │ weight = weight.view(torch.int16) │ │ 36 │ │ dtype = "bfloat16" │ │ ❱ 37 │ array = weight.cpu().numpy() │ │ 38 │ tensor_file = os.path.join(offload_folder, f"{weight_name}.dat") │ │ 39 │ if index is not None: │ │ 40 │ │ if dtype is None: │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ NotImplementedError: Cannot copy out of meta tensor; no data!