search

BY571 / IQN-and-Extensions

PyTorch Implementation of Implicit Quantile Networks (IQN) for Distributional Reinforcement Learning with additional extensions like PER, Noisy layer, N-step bootstrapping, Dueling architecture and parallel env support.
MIT License
81 stars 16 forks source link

Fix error of get_action in script version #7

Closed Jasonxu1225 closed 1 year ago

Jasonxu1225 commented 1 year ago

In the paper, we should use K to get action. So I think the script version is wrong because it use N to get_action instead of K. (Although the paper says that IQN Is not sensitive to the value of K). So I modify the IQN to get_action with K, which corresponding to the paper.

Jasonxu1225 commented 1 year ago

I find that I was wrong. Sorry.

Jasonxu1225 commented 1 year ago

After specific considerations, I think maybe the notebook version is correct, namely using K to get_action as a sample. And I modify the script version, where making IQN to get_action with K.

Jasonxu1225 commented 1 year ago

I have started a new PR. Please check it. Thank you very much!