OpenBMB / CPM-Live

Live Training for Open-source Big Models
511 stars 40 forks source link

CPM-Ant with bminf error #336

Closed havocio closed 1 year ago

havocio commented 1 year ago

1 GPU command : python3 text_generation.py --use-bminf --memory-limit 4 error : Traceback (most recent call last): File "text_generation.py", line 34, in model = bminf.wrapper(model, quantization=False, memory_limit=args.memory_limit << 30) File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 55, in wrapper model, found_linear = _wrapper(model, quantization, False, memory_limit) File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 30, in _wrapper model._modules[kw], fd = _wrapper(model._modules[kw], quantization, is_module_list or is_in_blocklist, memory_limit) File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 30, in _wrapper model._modules[kw], fd = _wrapper(model._modules[kw], quantization, is_module_list or is_in_blocklist, memory_limit) File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 34, in _wrapper model = TransformerBlockList([ File "/usr/local/lib/python3.8/dist-packages/bminf/scheduler/init.py", line 382, in init raise ValueError("Missing some parameters in layer %d" % i) ValueError: Missing some parameters in layer 2

单卡环境,不用bminf可以正常运行,尝试加上bminf就报如上错误,求教

zh-zheng commented 1 year ago

你是不是使用的压缩版模型?BMInf目前暂不支持压缩版的CPM-Ant模型。

havocio commented 1 year ago

明白了,感谢