Support virtual pipeline parallel in global batch size calculation

NVIDIA / bionemo-framework

BioNeMo Framework: For building and adapting AI models in drug discovery at scale

https://nvidia.github.io/bionemo-framework/

Other

180 stars 22 forks source link

Open jstjohn opened 1 month ago

jstjohn commented 1 month ago

@sichu2023, @pstjohn this might be important to get to good perf with the 100b model.