search

hughperkins / DeepCL

OpenCL library to train deep convolutional neural networks
Mozilla Public License 2.0
865 stars 199 forks source link

Python Sample of Double DQN, is it through multinet=2? #119

Open viper7882 opened 7 years ago

viper7882 commented 7 years ago

HI @hughperkins ,

Thank you for the Q-Learning sample you've provided in the Python directory. I'm looking to construct Double DQN using DeepCL where my goal is to play Atari Learning Environment (ALE) in Python 2.7 environment.

The closest example I could find in command line documentation, namely multinet allowing user to train several neural networks at the same time. Does it mean if I'm looking for Double DQN, I'll just have to configure multinet=2?

Even if that's the case, I couldn't find any Python API that provides multinet configuration. Have I missed out something or you have a plan to support this in the near future?

hughperkins commented 7 years ago

Possibly, but it wasnt designed exactly for that purpose: it was designed conceptually as an ensemble, to improve predictions. Which is similar to what you want. But you might find that the precise values you need for the Double-Q paper are not quite exposed currently, and you might need to tweak the code a bit to expose those , through the API.

Alternatively, you could simply train two nets, separately, and handle deriving the estimates you need from those.

I think that should be relatively straightforward, no global variables that will get in the way, cause race conditions etc, but you might want to just double-check this point. The fact that the multinet exists, and works, though, is good evidence that it should be possible to train two nets, in the same process, without conflicts.

(PS Thank you for the star on Coriander-dnn :-) )

viper7882 commented 7 years ago

Hi Hugh,

The pleasure is mine for Starring Coriander DNN.

I would agree the original Double DQN paper is rather a challenge to obtain. However if you google a few websites (such as my favourite, this and this), it is rather easy to figure out the primary content of the paper without knowing too much of details.

Using example provided by Mxnet, I could understand your suggestion to train two nets separately is entirely doable.

Probably a suggestion for improvement in Python DeepCL would be to have a "clone" function where the net could be easily duplicated once the first net is formed.

My wishlist is still it would be great if you could provide an official example of Double DQN while I'm working on mine.