replaced call to `_prepare_decoder_attention_mask()` with `_prepare_4d_causal_attention_mask()`

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

https://petals.dev

MIT License

8.89k stars 490 forks source link

replaced call to `_prepare_decoder_attention_mask()` with `_prepare_4d_causal_attention_mask()` #545

Open poedator opened 7 months ago

poedator commented 7 months ago

fixes issue #536 edited calls to mask methods consistently with https://github.com/huggingface/transformers/pull/27086

justheuristic commented 7 months ago

Hi!

Note: a day ago, we had suspicions that the issue is with the main branch and not this one. It appears not to be the case.

I can confirm that def test_input_ids, specifically, passes in the main branch but fails here. This is most likely a bug. I can look into this eventually, but if you have time, please look into this yourself first.