search

gaozhihan / PreDiff

[NeurIPS 2023] Official implementation of "PreDiff: Precipitation Nowcasting with Latent Diffusion Models"
Apache License 2.0
90 stars 5 forks source link

Training Time of Prediff #12

Open yyyujintang opened 9 months ago

yyyujintang commented 9 months ago

Could you provide the actual training time cost of the three models? According to the appendix, VAE, LDM, and KCN need 200, 1000, and 200 training epochs, respectively.

I tried on 3090 Ti and the experiment time costs are as follows. <!DOCTYPE html> 4*3090 Epochs 1gpu 2gpu 4gpu 4gpu Estimation (Day)
VAE-338MB 200 2h 1h 52min 1h20min 11.11
LDM-548MB 1000 2h 1h16min 53min 36.81
KCN-35MB 200 30min 15min 8min 1.11
gaozhihan commented 9 months ago

Thank you for your interest in our work and your question. It takes around 12 days for training the VAE on 4 NVIDIA A10G GPUs. It takes around 32 days for training the latent Earthformer-UNet using the same GPU configs.

yyyujintang commented 9 months ago

Thank you for your reply. It is very helpful.