search

miyosuda / async_deep_reinforce

Asynchronous Methods for Deep Reinforcement Learning
Apache License 2.0
592 stars 192 forks source link

Why you are use custom gradint appiler? #21

Open dm-mch opened 7 years ago

dm-mch commented 7 years ago

Hello, thank you for your code! Why you are not use standart apply_gradients for rms optimizer? Any sync issue for multithreading? https://www.tensorflow.org/versions/master/api_docs/python/train/optimizers#Optimizer.apply_gradients

miyosuda commented 7 years ago

I created this class because I needed to calculate gradients with local loss and variables, but we need to apply calculated gradients to global variables. I also needed to share "rms" and "momentum" slots among threads, to do SharedRMSProp.

I chose to create my own class, but it might be possible to use standard RMSPropOptimizer, if we call compute_gradients() with local loss and variables, and call apply_gradients with calculated gradients and global variables.