modeling_bloom.py787行报TypeError: 'NoneType' object is not subscriptable

[xzh12890]

界面处理执行推理时报 File "/mnt/workspace/TigerBot/apps/modeling_bloom.py", line 787, in forward outputs = block( File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, *kwargs) File "/mnt/workspace/TigerBot/apps/modeling_bloom.py", line 440, in forward attn_outputs = self.self_attention( File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, kwargs) File "/mnt/workspace/TigerBot/apps/modeling_bloom.py", line 291, in forward fused_qkv = self.query_key_value(hidden_states) # [batch_size, seq_length, 3 x hidden_size] File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(args, kwargs) File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/bitsandbytes/nn/modules.py", line 402, in forward out = bnb.matmul(x, self.weight, bias=self.bias, state=self.state) File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 562, in matmul return MatMul8bitLt.apply(A, B, out, bias, state) File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/torch/autograd/function.py", line 506, in apply return super().apply(*args, **kwargs) # type: ignore[misc] File "/home/pai/envs/bqgpt/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 383, in forward outliers = state.CB[:, state.idx.long()].clone() TypeError: 'NoneType' object is not subscriptable

[Tlntin]

拉一下最新代码再试试。 顺便问一下执行了什么命令

[xzh12890]

采用最新的代码还是一样。就是CUDA_VISIBLE_DEVICES=0 python ./apps/web_demo.py以后，正常问问题

[Tlntin]

CUDA_VISIBLE_DEVICES=0 python ./apps/web_demo.py

我测试了一下貌似没有报错，你可以试试把这个仓库的最新代码下载下来替换一下你的项目里面的代码。

[Tlntin]

或者你也可以试试部署api，再启动web_api_demo.py，效果是一样的。