[BUG] validate_yaml() isn't in sync with arguments check

NVIDIA / Megatron-LM

Ongoing research training transformer models at scale

Other

10.62k stars 2.38k forks source link

Open pierric opened 1 day ago

pierric commented 1 day ago

Describe the bug This check isn't in sync with the one in the arguments.py

To Reproduce

in yaml config, pipeline_model_parallel_size = 2, and num_layers_per_virtual_pipeline_stage = 2
pass --yaml-cfg

Expected behavior The setting pipeline_model_parallel_size = 2 should be accepted.

Proposed fix sync up the check in the two places.