issues
search
huggingface
/
nanotron
Minimalistic large language model 3D-parallelism training
Apache License 2.0
1.14k
stars
107
forks
source link
Fixing testsuite
#8
Closed
3outeille
closed
9 months ago
3outeille
commented
9 months ago
Tolerance tuning
fixing imports and some tests (averaging loss across nb of minibatch, concurrency problem, dict)
speeding up testsuite from ~1h => ~10 min (use more workers + tests only up to 4 gpus)