运行 gen.sh 脚本的时候报错

在运行gen.sh 脚本的时候报错，模型使用的是longlm-small, 指向命令 gen = model.generate(input_ids, do_sample=True, max_length=512, top_k=40, temperature=0.7, decoder_start_token_id=1)

如果将这一行替换成 gen = model.generate(input_ids, do_sample=True, decoder_start_token_id=1, top_p=0.9, max_length=512)

或者将temperature参数去除则不会出现个错误。

请问，直接命令 gen = model.generate(input_ids, do_sample=True, decoder_start_token_id=1, top_p=0.9, max_length=512) 使用来跑outgen的task，会不会损失一些模型的性能

Traceback (most recent call last):
  File "./gen.py", line 79, in <module>
    gen = model.generate(input_ids, do_sample=True, max_length=512, top_k=40, temperature=0.7, decoder_start_token_id=1)
  File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/generation_utils.py", line 1073, in generate
    return self.beam_sample(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/generation_utils.py", line 2114, in beam_sample
    next_tokens = torch.multinomial(probs, num_samples=2 * num_beams)
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

thu-coai / LOT-LongLM

运行 gen.sh 脚本的时候报错 #7