Where is the code for video saving?

Eclectic-Sheep / sheeprl

Distributed Reinforcement Learning accelerated by Lightning Fabric

https://eclecticsheep.ai

Apache License 2.0

274 stars 26 forks source link

Where is the code for video saving? #211

Closed LYK-love closed 3 months ago

LYK-love commented 4 months ago

Hello, I wonder where is the code for sheeprl to save videos (both training videos and testing videos). I noticed that, after training sheeprl (exp=DreamerV3), I have:

...
├── test_videos
│   └── rl-video-episode-0.mp4
└── train_videos
    ├── rl-video-episode-0.mp4
    ├── rl-video-episode-125.mp4
    ├── rl-video-episode-1.mp4
    ├── rl-video-episode-216.mp4
    ├── rl-video-episode-27.mp4
    ├── rl-video-episode-64.mp4
    └── rl-video-episode-8.mp4

in my logs/runs/... folder. But I can't find the program doing it in dreamer_v3.py. Meanwhile, I wonder is there a way to look at the video generated by the Decoder of DreamerV3. It should be blurry since it's reconstructed from compressed data, being able to look at it would be helpful for research.

belerico commented 4 months ago

Hi @LYK-love, the video generation is done by the gymnasium wrapper gym.experimental.wrappers.RecordVideoV0.

Meanwhile, I wonder is there a way to look at the video generated by the Decoder of DreamerV3. It should be blurry since it's reconstructed from compressed data, being able to look at it would be helpful for research.

We don't have it right now in the repo, but we can definitely share some notebooks that we have used in the past; in the meantime we can come up with a solution to also let the user be able to generate videos from the Dreamers family.

cc @michele-milesi

LYK-love commented 4 months ago

I see, thanks for your help.

michele-milesi commented 4 months ago

Hi @LYK-love, here you can find a notebook for reconstructing the RGB observations with Dreamer V3. I used the last version of SheepRL (v0.5.3).

I still have some comments to add, but only four things need to be defined:

The checkpoint path.
initial_steps: the number of steps played by the agent (set according to your environment and which part of the episode you want to analyze).
imagination_steps: the number of imagination steps you want to perform (must be lower than or equal to the initial_steps. If you are not interested in studying the imagination part, then feel free to set it equal to the initial_steps.
If you want to observe how the agent imagines future steps, you can decide to sample the actions from the agent or use the played actions.

I will let you know as soon as I have added some comments to the notebook.

michele-milesi commented 4 months ago

@LYK-love, I added some comments to the notebook. Feel free to ask questions if something is not clear.

LYK-love commented 4 months ago

@LYK-love, I added some comments to the notebook. Feel free to ask questions if something is not clear.

Sure. I'm worried about the performance of your implementation of DreamerV3 is not as well as the original one (by Hafner). Can you show some evaluation scores of DreamerV3?

belerico commented 4 months ago

Hi @LYK-love, if the notebook that @michele-milesi shared answers your question about generating videos from Dreamer-V3 we can close this issue and open a new one regarding the performances. Thank you