search

muupan / dqn-in-the-caffe

An implementation of Deep Q-Network using Caffe
MIT License
213 stars 118 forks source link

A doubt at line 350 in dqn.cpp #5

Closed chiggum closed 9 years ago

chiggum commented 9 years ago

Firstly, sorry I posted this as an issue as I wasn't able to find your mail.

Doubt: I am not able to understand whether the solver takes a million steps at solver_->step(1) as mentioned in dqn_solver.txt or only a single step. According to me 1 step == forward propagate, calculate errors, backpropagate errors and update filter weights.

Can you please clarify this? Sorry I wasn't able to get this part of caffe documentation.

Thanks! Regards.

muupan commented 9 years ago

No problem. Feel free to ask any question here.

solver_->Step(1) is a single step, an update from a minibatch of 32 transitions.

chiggum commented 9 years ago

Thanks a lot! I am also training an agent (rom: breakout) but facing quite a lot of difficulties as I am not using caffe but cudnn library. I'll then be asking you, my further queries here only. I hope to get some help.

Thanks a lot again. Regards.

chiggum commented 9 years ago

Hi again, As I said earlier, I am using breakout as the rom. I have written some code and it's working fine but unfortunately it's not learning as the slider in the game(the agent) either gets stuck to the left end of the screen or the right end.

After a lot of struggle, I am still unable to understand the reason for that. Do you have any clue why's that happening or can you have a look at my code please. I have added you as a collaborator in my private repo.

Thanks! Regards.

muupan commented 9 years ago

I've not read your code yet, but the discussion of this thread can be of some help https://groups.google.com/forum/#!topic/deep-q-learning/k1CzQxOi_v4

chiggum commented 9 years ago

Thanks a lot for that link! I think I can make it up from here.