Closed jianxing1 closed 4 months ago
No, it only handles spawning the process. You’d need to pass the other arguments into the Accelerator using the related classes and arguments (DeepspeedPlugin for deepspeed, mixed_precision=“bf16”
for your case here)
Is there a reason to not just use accelerate launch
?
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
System Info
Information
Tasks
no_trainer
script in theexamples
folder of thetransformers
repo (such asrun_no_trainer_glue.py
)Reproduction
Hello,
When I ran the official routine
nlp_example.py
, theREADME.md
mentioned that it can be run throughaccelerate launch
andtorchrun
.After I went through
accelerate config
and set the mixed precision tobf16
, I ranaccelerate launch
and it printed that the precision was indeedbf16
.But when I run
torchrun
, its mixed precision isno
.Does
torchrun
support automatic reading the yaml content ofaccelerate config
?Or how to enable
torchrun
to use the configuration information ofaccelerate config
?This is my
accelerate config
process:Expected behavior
I hope that the
torchrun
command can obtain the parameters in theaccelerate config
, includingmixed precision
, etc.