Closed havocio closed 1 year ago
1 GPU command : python3 text_generation.py --use-bminf --memory-limit 4 error : Traceback (most recent call last): File "text_generation.py", line 34, in model = bminf.wrapper(model, quantization=False, memory_limit=args.memory_limit << 30) File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 55, in wrapper model, found_linear = _wrapper(model, quantization, False, memory_limit) File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 30, in _wrapper model._modules[kw], fd = _wrapper(model._modules[kw], quantization, is_module_list or is_in_blocklist, memory_limit) File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 30, in _wrapper model._modules[kw], fd = _wrapper(model._modules[kw], quantization, is_module_list or is_in_blocklist, memory_limit) File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 34, in _wrapper model = TransformerBlockList([ File "/usr/local/lib/python3.8/dist-packages/bminf/scheduler/init.py", line 382, in init raise ValueError("Missing some parameters in layer %d" % i) ValueError: Missing some parameters in layer 2
单卡环境,不用bminf可以正常运行,尝试加上bminf就报如上错误,求教
你是不是使用的压缩版模型?BMInf目前暂不支持压缩版的CPM-Ant模型。
明白了,感谢
1 GPU command : python3 text_generation.py --use-bminf --memory-limit 4 error : Traceback (most recent call last): File "text_generation.py", line 34, in
model = bminf.wrapper(model, quantization=False, memory_limit=args.memory_limit << 30)
File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 55, in wrapper
model, found_linear = _wrapper(model, quantization, False, memory_limit)
File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 30, in _wrapper
model._modules[kw], fd = _wrapper(model._modules[kw], quantization, is_module_list or is_in_blocklist, memory_limit)
File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 30, in _wrapper
model._modules[kw], fd = _wrapper(model._modules[kw], quantization, is_module_list or is_in_blocklist, memory_limit)
File "/usr/local/lib/python3.8/dist-packages/bminf/wrapper.py", line 34, in _wrapper
model = TransformerBlockList([
File "/usr/local/lib/python3.8/dist-packages/bminf/scheduler/init.py", line 382, in init
raise ValueError("Missing some parameters in layer %d" % i)
ValueError: Missing some parameters in layer 2
单卡环境,不用bminf可以正常运行,尝试加上bminf就报如上错误,求教