Open moussaKam opened 2 months ago
Hi @moussaKam,
I'll open a PR with a fix soon, but meanwhile you can try applying the following commit yourself
https://github.com/swiss-ai/nanotron/commit/664c09aa48204b8a45756e15fac9cc6bf0b38ccf
Toni
Thanks @TJ-Solergibert
After running the toy example I run it again to resume training and I'm getting an error only if PP > 1
Here's the config:
Here's the error: