bionemo: bn2/add pipelineparallel dtype

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html

Apache License 2.0

10.82k stars 2.26k forks source link

bionemo: bn2/add pipelineparallel dtype #9475

Closed skothenhill-nv closed 2 weeks ago

skothenhill-nv commented 2 weeks ago

Fixes a bug from parallel_dtype not being correctly passed in the parallelism method.