search

simoninithomas / Deep_reinforcement_learning_Course

Implementations from the free course Deep Reinforcement Learning with Tensorflow and PyTorch
http://www.simoninithomas.com/deep-rl-course
3.74k stars 1.23k forks source link

No usage of stacked_frames #37

Closed sven-hoek closed 5 years ago

sven-hoek commented 5 years ago

I'm not really familiar with Python notebooks but from the code that's visible in both the Doom and the Space Invaders code, it seems that the stacked_frames are only updated but never used as input (in the memory only the actual, not even preprocessed, states/frames are being stored).

Couldn't we just store the stacked_frames instead of the frames and the rest (action, reward, next_state, done) can stay the same (the next_state could also be preprocessed, I guess)? Gonna try it, once I find the time.

HemaZ commented 5 years ago

stacked frame function is being used everywhere, check the memory pre-populate part of code.

state = game.get_state().screen_buffer
state, stacked_frames = stack_frames(stacked_frames, state, True)

here the current state is the returned one from stack_frames not the original.

also for the next_state, this is done.

next_state = game.get_state().screen_buffer
next_state, stacked_frames = stack_frames(stacked_frames, next_state, False)

and all of this is done also during the training.

sven-hoek commented 5 years ago

Oh yeah, true. Seems like it got a little too late yesterday...was already wondering, why no one else complained about it. Thanks a lot.