search

autonomousvision / giraffe

This repository contains the code for the CVPR 2021 paper "GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields"
https://m-niemeyer.github.io/project-pages/giraffe/index.html
MIT License
1.24k stars 160 forks source link

Optimizer choice #30

Closed AustinXY closed 3 years ago

AustinXY commented 3 years ago

Thank you for sharing this great work. I've been doing some experiments with the code, but I've noticed that the model would refuse to train if I change the optimizer to Adam (I've only tried on the LSUNCAR dataset, and when using RMSprop the model trains well). Is there any reason on why you choose to use RMSprop? And do you know why Adam would fail? Thanks.

m-niemeyer commented 3 years ago

Hi @AustinXY , thanks for your interest in our work. In previous works, we have often used RMSProp because Adam with default settings is often not stable in GAN settings. I haven't tried this, but using other hyperparameters (in partiular beta1=0, beta2=0.99) might resolve this, though!