search

danijar / dreamerv2

Mastering Atari with Discrete World Models
https://danijar.com/dreamerv2
MIT License
898 stars 195 forks source link

How many environment steps per update? #29

Closed mctigger closed 2 years ago

mctigger commented 2 years ago

Hi danijar, how many environment steps are you running per update? In the paper it is 4 (so after every step the agent makes it is updated because of action repeat?), but here in the config it says train_every: 16. What is the correct number?

Best, Tim

danijar commented 2 years ago

Hi Tim, those are both correct. It's 16 env steps or 4 agent steps. The action repeat introduces the factor of 4 between the two.

mctigger commented 2 years ago

Thank you for answering so quickly!

In the paper in the hyperparameters section it says Environment steps per update: 4. So in the paper it should actually be Agent Steps per update: 4 or Environment steps per update: 16? Just want to make sure I understand you correctly.

danijar commented 2 years ago

Ah, yes. It's every 16 frames or 4 actions. I'll update the paper to make it clearer.