The simulation screen always stops at the beginning of the third episode

[Exasor]

I run this env through python experiments.py evaluate configs/HighwayEnv/env.json configs/HighwayEnv/agents/DQNAgent/dqn.json --train --episodes=5 and the first 2 episodes are displayed normally but the simulation screen always stops at the beginning of the third episode, I don't know where this issue comes from and sincerely look for help

[eleurent]

Hi @Exasor, This is by design, and the reason is simple: rendering the screen takes more time than simply stepping the simulation, so to train faster by default we only render a small fraction of the episodes while training.

On the contrary, with the --test option, every episode is rendered by default.

If you want to change this behavior, you must change the video_callable argument in https://github.com/eleurent/rl-agents/blob/master/rl_agents/trainer/evaluation.py#L74

By default (None value), it is set here to capped_cubic_video_schedule, a method that only renders episodes in a cubic progression and then every 1000s, i.e. episodes: 1, 2, 8, 27, 64, ..., 1000, 2000, 3000, etc.

But you can set it to any function, for instance lambda episode: True to render all episodes.

[Exasor]

Oh I get it! Thanks for your reply

[Exasor]

Sorry to trouble you again, I have installed pytorch-gpu, so I wonder if this program can utilize gpu to train agent by default or I should make some settings

[eleurent]

Hi again, The simulator itself runs in CPU. But agents can be trained on GPU, for example when they use with neural network function approximation. The settings will depend on which RL library you are using. If you use https://github.com/eleurent/rl-agents/, then GPU is preferred by default if you train a DQN agent. I think GPU is also chosen by default in https://github.com/hill-a/stable-baselines.

[Exasor]

Thanks a lot