Refactoring tying mechanism + small fixes

huggingface / nanotron

Minimalistic large language model 3D-parallelism training

Apache License 2.0

1.14k stars 107 forks source link

Closed NouamaneTazi closed 7 months ago

NouamaneTazi commented 7 months ago

This PR is a preparation for a next PR to remove module_id_to_prefix from nanotron

✅ tied weights tests pass