search

microsoft / ProDA

Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021)
https://arxiv.org/abs/2101.10979
MIT License
286 stars 44 forks source link

The time of training #1

Closed super233 closed 3 years ago

super233 commented 3 years ago

How long will each stage last based on the paper setting? I am training the stage1 on 2 Tesla V100-DGXS-32GB, and it takes 2 hours to train 1000 iters(about 1.3 epoch), it's a little slow, is this to be expected?

panzhang0212 commented 3 years ago

It will take 36 hours in 4 Tesla V100-DGXS-32GB. Below is the training log: run_2020_12_01_06_27_02.log As seen from the log, you can train fewer epochs if you want to get final result quickly.

super233 commented 3 years ago

Thanks, and how about the stage2 and stage3?

panzhang0212 commented 3 years ago

stage2 log: run_2020_11_09_15_38_50.log stage3 log: run_2020_11_26_08_50_10.log

ljjcoder commented 2 years ago

@panzhang0212 It takes 36 hours for each stage, 108 hours in total right?