Update ignore_index parameter for flash attention

allenai / OLMo

Modeling, training, eval, and inference code for OLMo

https://allenai.org/olmo

Apache License 2.0

4.24k stars 400 forks source link

Update ignore_index parameter for flash attention #580

Closed 2015aroras closed 2 months ago

2015aroras commented 2 months ago

The flash attention library changed the cross_entropy_loss parameter ignored_index to ignore_index in v2.5.8. This changes lets us support v2.5.8.