baichuan-inc / Baichuan2

A series of large language models developed by Baichuan Intelligent Technology
https://huggingface.co/baichuan-inc
Apache License 2.0
4.09k stars 295 forks source link

RuntimeError: probability tensor contains either inf, nan or element < 0 #315

Open HiXiaochen opened 11 months ago

HiXiaochen commented 11 months ago

在A100上进行batch测试时没有问题,换到A10上会报错,打印出来next token logits,padding部分对应的输出值是nan。采用的是left padding,pad值为0,请问这是什么原因导致的呢? infer代码: messages = ['' + f"原始输入文本" + '' for data in messages] res = tokenizer(messages, padding=True) input_ids_list = res['input_ids'] input_ids = torch.LongTensor(input_ids_list).to(model.device) outputs = model.generate(input_ids=input_ids)

报错:

image
blueskyban commented 9 months ago

你好,我在A100上跑,cli_demo时就报probability tensor contains either inf, nan or element < 0这个错误了,你的问题解决了么,你在cli_demo时是正常的?

shichaoYan commented 8 months ago

你好,我现在在A10上遇到了同样的问题,请问你的错误解决了吗?