hardmaru / WorldModelsExperiments

World Models Experiments
608 stars 171 forks source link

Memory leak and potential fix #13

Closed davidsandberg closed 5 years ago

davidsandberg commented 5 years ago

Hi @hardmaru! Thanks for a great repo and a really cool project!! I was running the 'carracing' experiment but experienced memory problems. This was caused by new tensorflow assign operations being created every time set_model_params (in rnn.py and vae.py) is called. I implemented a fix where the assign operations were created when the graph was built and used when setting the parameters. If you are interested in a fix I can make a PR, or you simply just copy and paste the fix from here (look at the diffs in rnn.py and vae.py).

hardmaru commented 5 years ago

Thanks for identifying the leak! Can you do a favor and create a PR?

On Wed, Dec 26, 2018 at 6:41 AM David Sandberg notifications@github.com wrote:

Hi @hardmaru https://github.com/hardmaru! Thanks for a great repo and a really cool project!! I was running the 'carracing' experiment but experienced memory problems. This was caused by new tensorflow assign operations being created every time set_model_params (in rnn.py and vae.py). I implemented a fix where the assign operations were created when the graph was built and used when setting the parameters. If you are interested in a fix I can make a PR, or you simply just copy and paste the fix from here https://github.com/davidsandberg/WorldModelsExperiments/compare/c31e8d84bcfdbebf12513ccdef9800d20477542d...e759ffc140925b324b6fac28f9e23cc1789709bc (look at the diffs in rnn.py and vae.py).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hardmaru/WorldModelsExperiments/issues/13, or mute the thread https://github.com/notifications/unsubscribe-auth/AGBoHui1d97sshNhAUMzvyWuMAUD9yuhks5u8puGgaJpZM4ZhOVY .