DQN

[Het-Shah]

Add/remove the variants we should implement

• [x] DQN
• [x] Double Q Learning
• [x] Dueling DQN
• [x] Prioritized Replay
• [x] Noisy Nets (Noisy DQN)
• [x] Distributional DQN
• [ ] Adapt multi step bootstrapped return from A3C

Refer #6

[Sharad24]

I think it would be nice to move towards implementing a Rainbow DQN type of formulation. Since that is the best DQN

[Het-Shah]

Should we leave this to the person using this? Like having options for individual implementations and then also having Rainbow DQN.

[Sharad24]

We can have one single implementation with these options being selected based on arguments in the constructor. E.g.

DQN(env, noisy=False, target_networks=False, etc)

What do you think?

[Het-Shah]

Yea that is what I was saying.. Having all the implementations and rainbow as well in one place.

[Het-Shah]

Just add to the checklist all the variants according to you...

[Sharad24]

Sounds good! Just updated.

Some other info regarding this, it seems to me that having the Noisy Layers as separate neural network modules might be better than directly defining inside DQN. This way we can just call the Noisy Layer like any other pytorch nn module.

[Het-Shah]

Yup will add that in utils.py mostly.

[ajaysub110]

@Het-Shah when you're working on DQN, make sure that it is similar in structure to DDPG since their methods will be used in general by HER and TD3

[Het-Shah]

Ok, Sure.