search

carpedm20 / deep-rl-tensorflow

TensorFlow implementation of Deep Reinforcement Learning papers
MIT License
1.6k stars 396 forks source link

questions about the Dueling logic in network.py #6

Closed liurida closed 8 years ago

liurida commented 8 years ago

Hi It's really a good code for learning Reinforcement Learning. In the network.py, I have 2 questions.

  1. I think you want to assert len(value_hidden_sizes) != 0 and len(advantage_hidden_sizes) != 0.
  2. About the Dueling part, the logic in code is layer contains value_hidden_sizes linear, then the layer is delivered to the next advantage logic. But I read the related paper, if I understand correct, it describes that the state-value and advantage are generated from the same source observation, then they're added together, and minus the mean advantage value.

Looking forward to your further response.

carpedm20 commented 8 years ago

@liurida Thanks! You're totally right. That's a huge bug and I just fixed and commit the new code.