recorded mp4 format video cannot play

[JianmingTONG]

I run the following code to start training progress.

Under the path of the script directory

python3 experiments.py evaluate configs/HighwayEnv/env_medium.json configs/HighwayEnv/agents/DQNAgent/ddqn.json --train --episodes=1000000

However, the recorded video cannot be played, as attached.

https://user-images.githubusercontent.com/30002985/103257097-c4dd7000-49ca-11eb-925a-c1ae7069ad47.mp4

Might I request anyone to help?

[eleurent]

That is very strange. Do you have ffmpeg installed? Did the first episode fully complete? (the video is only "closed" and saved at the end)

[JianmingTONG]

Thanks for the reply.

Yes, I do install the ffmpeg. I currently use imageio instead as following:

import imageio

images = []
images.append(env.render(mode='rgb_array'))
imageio.mimsave("test.gif", images, "GIF", 0.1) 

The gif looks not so smooth like video rendered in google colab. Therefore, I am also looking forward to some solution to it.

[eleurent]

The problem is with this technique is that env.render() only produces a frame when we take an action, which happens at 1Hz, while the simulation itself is run at 15Hz (by default). So the resulting video only contains snapshots of every second and will be jerky.

The (hacky) way that I found to circumvent this is to provide a rendering callback to the environment viewer, so that intermediate frames can be rendered and saved to a video when the simulation is propagated. Here is how your snippet should be adapted:

import imageio

images = []
capture_frame = lambda: images.append(env.render(mode='rgb_array'))
env.automatic_rendering_callback = capture_frame

for _ in range(3):
    env.render()
    env.step(env.action_space.sample())

imageio.mimsave("test.gif", images, "GIF")

[eleurent]

See also https://github.com/eleurent/highway-env/issues/118

[JianmingTONG]

Thanks!