Open jeffxtang opened 1 year ago
I followed the fine-tune-code-llama.ipynb and got an error when running trainer.train():
File ~/anaconda3/envs/codallama/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py:50, in make_causal_mask(input_ids_shape, dtype, device, past_key_values_length) 46 """ 47 Make causal mask used for bi-directional self-attention. 48 """ 49 bsz, tgt_len = input_ids_shape ---> 50 mask = torch.full((tgt_len, tgt_len), torch.finfo(dtype).min, device=device) 51 mask_cond = torch.arange(mask.size(-1), device=device) 52 mask.masked_fill(mask_cond < (mask_cond + 1).view(mask.size(-1), 1), 0)
RuntimeError: value cannot be converted to type at::Half without overflow
My transformers version is 4.34.0.dev0 and accelerate is >=0.23.0. Any ideas? Thanks!
I followed the fine-tune-code-llama.ipynb and got an error when running trainer.train():
File ~/anaconda3/envs/codallama/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py:50, in make_causal_mask(input_ids_shape, dtype, device, past_key_values_length) 46 """ 47 Make causal mask used for bi-directional self-attention. 48 """ 49 bsz, tgt_len = input_ids_shape ---> 50 mask = torch.full((tgt_len, tgt_len), torch.finfo(dtype).min, device=device) 51 mask_cond = torch.arange(mask.size(-1), device=device) 52 mask.masked_fill(mask_cond < (mask_cond + 1).view(mask.size(-1), 1), 0)
RuntimeError: value cannot be converted to type at::Half without overflow
My transformers version is 4.34.0.dev0 and accelerate is >=0.23.0. Any ideas? Thanks!