search

NVIDIA / Megatron-LM

Ongoing research training transformer models at scale
https://docs.nvidia.com/megatron-core/developer-guide/latest/user-guide/index.html#quick-start
Other
10.18k stars 2.29k forks source link

Vit Classify #648

Closed vksastry closed 1 month ago

vksastry commented 9 months ago

I am trying to run examples/pretrain_vision_classify.sh. I am wondering if tensor parallelism and pipeline parallelism are supported for vision models ? In other words, can I use tensor-model-parallel-size or pipeline-model-parallel-size greater than 1?

github-actions[bot] commented 7 months ago

Marking as stale. No activity in 60 days.