Open gorjanradevski opened 1 month ago
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
System Info
Information
Tasks
no_trainer
script in theexamples
folder of thetransformers
repo (such asrun_no_trainer_glue.py
)Reproduction
Expected behavior
By running the code snippet above
CUDA_VISIBLE_DEVICES=0,1,6,7 accelerate launch --num_processes 4 scripts/test_fsdp.py
, I get:However, if I set
'fsdp_activation_checkpointing': False
, then no such error takes place.