Add a checkpoint averaging script for the new .distcp checkpoint format

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html

Apache License 2.0

12.32k stars 2.55k forks source link

Add a checkpoint averaging script for the new .distcp checkpoint format #10462

Open Kipok opened 2 months ago

Kipok commented 2 months ago

Is your feature request related to a problem? Please describe.

We found checkpoint averaging to be very helpful, but the current scripts in https://github.com/NVIDIA/NeMo/tree/main/scripts/checkpoint_averaging don't work with the new format.

Describe the solution you'd like

Please provide an example of the averaging script that works.

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] commented 1 month ago

This issue was closed because it has been inactive for 7 days since being marked as stale.