search

LeapLabTHU / EfficientTrain

1.5−3.0× lossless training or pre-training speedup. An off-the-shelf, easy-to-implement algorithm for the efficient training of foundation visual backbones.
MIT License
188 stars 8 forks source link

Command script bugs #11

Closed iminfine closed 2 weeks ago

iminfine commented 2 weeks ago

I am reproducing your results following the training instruction i.e using the command: CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \ python _ET_pp_main_deitS.py\ --tag /deit_S \ --epoch 200 \ --seed 0

The script generates the following: Please notice the epochs are wrong!

`res_list: [96, 96, 160, 160, 160, 160, 192, 192, 224, 224] bs_list: [512, 512, 512, 512, 512, 512, 256, 256, 256, 256] up_freq_list: [1, 1, 1, 1, 1, 1, 2, 2, 2, 2] replay_times_list: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1] replay_buffer_size_list: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 1088 --end_epoch 108         --warmup_epochs 217         --aa rand-m0-mstd0.5-inc1         --input_size 96         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 512 --lr 4e-3 --update_freq 1         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 1088 --end_epoch 217         --warmup_epochs 217         --aa rand-m1-mstd0.5-inc1         --input_size 96         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 512 --lr 4e-3 --update_freq 1         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 392 --end_epoch 117         --warmup_epochs 78         --aa rand-m2-mstd0.5-inc1         --input_size 160         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 512 --lr 4e-3 --update_freq 1         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 392 --end_epoch 156         --warmup_epochs 78         --aa rand-m3-mstd0.5-inc1         --input_size 160         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 512 --lr 4e-3 --update_freq 1         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 392 --end_epoch 196         --warmup_epochs 78         --aa rand-m4-mstd0.5-inc1         --input_size 160         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 512 --lr 4e-3 --update_freq 1         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 392 --end_epoch 235         --warmup_epochs 78         --aa rand-m5-mstd0.5-inc1         --input_size 160         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 512 --lr 4e-3 --update_freq 1         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 272 --end_epoch 190         --warmup_epochs 54         --aa rand-m6-mstd0.5-inc1         --input_size 192         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 256 --lr 4e-3 --update_freq 2         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 272 --end_epoch 217         --warmup_epochs 54         --aa rand-m7-mstd0.5-inc1         --input_size 192         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 256 --lr 4e-3 --update_freq 2         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 200 --end_epoch 180         --warmup_epochs 40         --aa rand-m8-mstd0.5-inc1         --input_size 224         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 256 --lr 4e-3 --update_freq 2         --replay_times 1 --replay_buffer_size 0         --seed 0

save at: output/deit_S

     CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7         python -m torch.distributed.launch --use-env --nproc_per_node=8 --master_port=11333 main_buffer.py         --data_path /datasets/image_net  --num_workers 10         --output_dir ./output/deit_S         --epochs 200 --end_epoch 200         --warmup_epochs 40         --aa rand-m9-mstd0.5-inc1         --input_size 224         --model deit_small_patch16_224 --drop_path 0.1         --use_amp true --clip_grad 5.0         --batch_size 256 --lr 4e-3 --update_freq 2         --replay_times 1 --replay_buffer_size 0         --seed 0   `
blackfeather-wang commented 2 weeks ago

Thanks for your attention. In fact, it's correct. Please refer to the definition of EfficientTrain++ in our paper.

iminfine commented 1 week ago

Thanks for your reply. I had reproduced the result of deit_S.