Thanks for sharing the nice work! May I ask a question for the training time?
I am training on the cluster of 1 PC with 8 GPU (NVIDIA A100), it takes 3 min per epoch for generator and 6 min per epoch for details predictor. I am using default config which is 3000 epochs. This means 3 days for generator and 6 days for details predictor. The paper says 3days for generator and 12 hours for predictor, which seems not matching my number. Any ideas on that? Also using 1 GPU and 8 GPU doesn't seem to change too much on the speed during my testing, a 50% improvement on 8gpu compared to 1 gpu for generator. do i miss anything?
Also it says 800k iteration for generator and 60k iterations for predictor, so how is 800k and 60k iterations are calculated?
Thanks for sharing the nice work! May I ask a question for the training time? I am training on the cluster of 1 PC with 8 GPU (NVIDIA A100), it takes 3 min per epoch for generator and 6 min per epoch for details predictor. I am using default config which is 3000 epochs. This means 3 days for generator and 6 days for details predictor. The paper says 3days for generator and 12 hours for predictor, which seems not matching my number. Any ideas on that? Also using 1 GPU and 8 GPU doesn't seem to change too much on the speed during my testing, a 50% improvement on 8gpu compared to 1 gpu for generator. do i miss anything?
Also it says 800k iteration for generator and 60k iterations for predictor, so how is 800k and 60k iterations are calculated?
thank you very much!