CodeShell-7B-Chat-int4 启动wed_demo.py发送信息时报错，AttributeError: 'list' object has no attribute 'absmax

[Coooolrui]

Exception in thread Thread-6 (generate): Traceback (most recent call last): File "/opt/conda/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/opt/conda/lib/python3.10/threading.py", line 953, in run self._target(*self._args, self._kwargs) File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "/opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py", line 1479, in generate return self.greedy_search( File "/opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py", line 2340, in greedy_search outputs = self( File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/CodeShell-7B-Chat-int4/modeling_codeshell.py", line 886, in forward transformer_outputs = self.transformer( File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/CodeShell-7B-Chat-int4/modeling_codeshell.py", line 694, in forward outputs = block( File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/root/.cache/huggingface/modules/transformers_modules/CodeShell-7B-Chat-int4/modeling_codeshell.py", line 382, in forward attn_outputs = self.attn( File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/CodeShell-7B-Chat-int4/modeling_codeshell.py", line 272, in forward query_states, key_states, value_states = self.c_attn(hidden_states).split((self.embed_dim, self.kv_dim, self.kv_dim), dim=2) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/opt/conda/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 256, in forward out = bnb.matmul_4bit(x, self.weight.t(), bias=bias, quant_state=self.weight.quant_state) File "/opt/conda/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 577, in matmul_4bit return MatMul4Bit.apply(A, B, out, bias, quant_state) File "/opt/conda/lib/python3.10/site-packages/torch/autograd/function.py", line 539, in apply return super().apply(*args, **kwargs) # type: ignore[misc] File "/opt/conda/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 516, in forward output = torch.nn.functional.linear(A, F.dequantize_4bit(B, quant_state).to(A.dtype).t(), bias) File "/opt/conda/lib/python3.10/site-packages/bitsandbytes/functional.py", line 1017, in dequantize_4bit absmax = quant_state.absmax AttributeError: 'list' object has no attribute 'absmax'

[ruixie]

我在本地使用了Codeshell-int4 版本，但是没有复现你的问题，你可以把你的wed_demo.py代码发上来，方便我们调试。

[VVIP1024]

这个问题是因为运行web_demo.py脚本的时候需要安装：bitsandbytes，最新版为：0.43.0，这个版本安装后就会有这个问题；可以将改模块回退两个版本：0.41.0，就可以正常运行了