使用SFT后的FP32模型进行生成，报错RuntimeError: where expected condition to be a boolean tensor, but got a tensor with dtype Half

OpenMOSS / MOSS

An open-source tool-augmented conversational language model from Fudan University

https://txsun1997.github.io/blogs/moss.html

Apache License 2.0

11.92k stars 1.14k forks source link

使用SFT后的FP32模型进行生成，报错RuntimeError: where expected condition to be a boolean tensor, but got a tensor with dtype Half #233

Open ARIELDENG opened 1 year ago

ARIELDENG commented 1 year ago

通过zero_to_fp32.py文件将上述多组pt文件转成pytorch_model.bin，此外在index.json里将所有参数都指向了pytorch_model.bin

hingkan commented 1 year ago

您好，我这里也出现了同样的问题，请问您解决了嘛。如解决了，能给个提示吗，谢谢

cjrzh commented 1 year ago

我也是这个问题，mark一下

hingkan commented 1 year ago

我比较简陋的在模型加载时指定的torch_dtype删除，如： raw_model = MossForCausalLM._from_config(config) model = load_checkpoint_and_dispatch( raw_model, model_path, device_map="auto", no_split_module_classes=["MossBlock"]) 如你们找到好的办法，希望告诉我，谢谢。