search

microsoft / CNTK

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
https://docs.microsoft.com/cognitive-toolkit/
Other
17.49k stars 4.3k forks source link

Deep Reinforcement Learning #2106

Open rems75 opened 6 years ago

rems75 commented 6 years ago

I’ve started playing a bit with the code at https://github.com/Microsoft/CNTK/blob/t-mofun/dqn/Examples/ReinforcementLearning/DeepQNeuralNetwork.py. First, thanks for the implementation, it’s definitely gonna be useful! I’ve encountered a small bug while training a mini-batch (line 399 of the file). Data sanitization is applied to the array “terminals” (containing booleans), and crashes with the message ‘only integer, float32 and float64 are supported’. I’ve had to modify my file core.py in the main library for it to run.

mfuntowicz commented 6 years ago

Hi @rems75,

What you report is very interesting and is not caught by the unit test for the DQN example ...

Can you give a little bit more context: CNTK version ? Gym version ?

Thanks in advance, Morgan

sayanpa commented 6 years ago

@rems75: Do you have a fix for the problem? If you do can you share the fix or issue a PR

rems75 commented 6 years ago

@sayanpa In the end I simply cast the booleans in the "terminals" array into integers, which worked. It might be worth adding a check or conversion though, since a state being terminal or not might be a boolean in certain environments.

sayanpa commented 6 years ago

Thanks @rems75: Would you mind submitting a small Pull Request? Or just attach the code here and I will merge the changes.