search

keon / deep-q-learning

Minimal Deep Q Learning (DQN & DDQN) implementations in Keras
https://keon.io/deep-q-learning
MIT License
1.29k stars 455 forks source link

Speeding the replay #18

Closed marxav closed 6 years ago

marxav commented 6 years ago

First, thank you for this wonderful code.

In the replay function, there is one model.fit(state, target_f) per sample in the minibach (i.e. if there are 32 samples, then there are 32 fit ).

I think all samples of the minibatch could be used in a single update with one single train_on_batch(states, targets_f), which would speed up the processing time.

pskrunner14 commented 6 years ago

@marxav you're right. Training on batches could speed up the process and result in stable convergence as well.

keon commented 6 years ago

yup. will do that someday if it is possible without hurting readability. I chose one by one update because it fits the purpose of the demonstration.