Is there any way to avoid using torchrun?

[guoqincode]

You can try wrapping train.py with accelerate to avoid torchrun and still get faster training and less memory