search

rlcode / reinforcement-learning

Minimal and Clean Reinforcement Learning Examples
MIT License
3.37k stars 728 forks source link

update target_model before loading saved model in cartpole_dqn.py #86

Open nyck33 opened 5 years ago

nyck33 commented 5 years ago 
# initialize target model with same weights as the model, in case we load a model
        #shouldn't this be done after load_model?
        self.update_target_model()

        if self.load_model:
            self.model.load_weights("./save_model/cartpole_dqn.h5")

could be

if self.load_model:
            self.model.load_weights("./save_model/cartpole_dqn.h5")

 self.update_target_model()

so that if we load a saved model, the target_model would have the saved weights rather than starting with the Keras-initialized weights.

I'm going to test this but it seems like the loaded model would be using an inferior target_model for at least the first episode and the model weights could get adjusted in the wrong way in that first episode, slightly slowing down it's learning.