search

keon / deep-q-learning

Minimal Deep Q Learning (DQN & DDQN) implementations in Keras
https://keon.io/deep-q-learning
MIT License
1.29k stars 455 forks source link

should update the weight every time step ? #17

Open fi000 opened 6 years ago

fi000 commented 6 years ago

should update the weight every time step ? (I think it is better to update the weight every for instance 10 steps in time step T/10==0 then saveweight) but in code it is updated for every 10 steps of episodes?

pskrunner14 commented 6 years ago

@fi000 actually since we're randomly sampling a batch from memory at each of these time steps, it would essentially only decrease the number of iterations/ batch updates. I'm not very clear on whether you're referring to weight updates or weight saves.

fi000 commented 6 years ago

Thank @pskrunner14, 1- I has applied this code to my problem and I saw that loading the weight is not useful at all cases and leads to divergence! What we can say about this? 2- Also, what is normally iteration steps to save the weights?As the low amount is not useful and higher amount leads to few saving

WorksWellWithOthers commented 3 years ago
  1. Loading just the weights of the model then sampling from a new set of experiences may be leading to divergence.
  2. How about saving after each episode? How about saving when there is an improvement?