Closed ShibataGenjiro closed 4 years ago
Hello!
I want to know how long will take to train the model with only one gpu?
According to other questions, https://github.com/nlpyang/BertSum/issues/47 It will take about 12 hours in 3 gpus.
But I only spent about 11 hours for training the model (BERTSUM+Classifier) after 50000 steps in only one 1080Ti GPU. Is it normal?
My command: python train.py -mode train -encoder classifier -dropout 0.1 -bert_data_path ../bert_data/cnndm -model_path ../models/bert_classifier -lr 2e-3 -visible_gpus 0 -gpu_ranks 0 -world_size 1 -report_every 50 -save_checkpoint_steps 1000 -batch_size 3000 -decay_method noam -train_steps 50000 -accum_count 2 -log_file ../logs/bert_classifier -use_interval true -warmup_steps 10000
python train.py -mode train -encoder classifier -dropout 0.1 -bert_data_path ../bert_data/cnndm -model_path ../models/bert_classifier -lr 2e-3 -visible_gpus 0 -gpu_ranks 0 -world_size 1 -report_every 50 -save_checkpoint_steps 1000 -batch_size 3000 -decay_method noam -train_steps 50000 -accum_count 2 -log_file ../logs/bert_classifier -use_interval true -warmup_steps 10000
同问
muximuxi
commented
2 years ago muximuxi
commented
2 years ago
Hello!
I want to know how long will take to train the model with only one gpu?
According to other questions, https://github.com/nlpyang/BertSum/issues/47 It will take about 12 hours in 3 gpus.
But I only spent about 11 hours for training the model (BERTSUM+Classifier) after 50000 steps in only one 1080Ti GPU. Is it normal?
My command:
python train.py -mode train -encoder classifier -dropout 0.1 -bert_data_path ../bert_data/cnndm -model_path ../models/bert_classifier -lr 2e-3 -visible_gpus 0 -gpu_ranks 0 -world_size 1 -report_every 50 -save_checkpoint_steps 1000 -batch_size 3000 -decay_method noam -train_steps 50000 -accum_count 2 -log_file ../logs/bert_classifier -use_interval true -warmup_steps 10000