CUDA error??????

[xvolcano02]

When I tried to use chatglm3-6b to test on LongBench, I got the following error after loading the model:

"variance = hidden_states.to(torch.float32).pow(2).mean(-1, keepdim=True) RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions."

Could someone help me?

[yyfcc17]

遇到同样问题，请问你是如何解决的？ @xvolcano02

[yyfcc17]

看起来是模型的最大长度不能设置为32768，要和那个json文件中一样设置成31500