How should I build a DQN with CNTK

[TheNetAdmin]

Is it possible to create a deep Q-Network with CNTK using Python API / C++ API? Since I've not discovered any DQN code with CNTK yet.

[n17s]

Yes it should be possible. The reinforcement learning python notebook on the Tutorials folder shows how to use the gym package from OpenAI. You can just try a different environment, including Atari environments. Of course you will have to use a convnet just like in the DQN paper from DeepMind but in principle, it is not much different from the python notebook tutorial (except it takes much longer to learn).

[MattChanTK]

Here is the link to that DQN tutorial: https://github.com/Microsoft/CNTK/blob/v2.0.beta3.0/bindings/python/tutorials/CNTK_203_Reinforcement_Learning_Basics.ipynb

I tried it and it worked pretty well, though it's not really "deep" though; just a normal neural network. https://medium.com/@tuzzer/cart-pole-balancing-with-q-learning-b54c6068d947#.56i7v3yaz

[Vladimir84]

This post has some additional explanation of that tutorial. Looks like it's up to you to implement the reinforcement learning part of the algorithm.

[n17s]

@tuzzer Yes the tutorial uses a shallow network. You can easily replace it with a deeper one and not much should change (except you need to wait more :-) @Vladimir84 Correct, there is no convenience functionality for RL right now. But all the ingredients to create the algorithms are there.