search

eloialonso / diamond

DIAMOND (DIffusion As a Model Of eNvironment Dreams) is a reinforcement learning agent trained in a diffusion world model. NeurIPS 2024 Spotlight.
https://diamond-wm.github.io
MIT License
1.53k stars 101 forks source link

The video show an abnormality after the character turn, is that expected? #23

Closed bailehang closed 3 weeks ago

bailehang commented 3 weeks ago

First of all, I would like to thank the author for his work, but after 38 seconds of the video, it is abnormal and can not be played, is that expected?

https://github.com/user-attachments/assets/a3bd22ca-a3c3-48d0-8702-f21793e59143

eloialonso commented 3 weeks ago

Hi @bailehang, yes we observed this kind of behavior too! These are autoregressive generations, so the accumulating error can cause the model to be out of distribution. It was actually quite surprising to us how long the generation manages to remain stable.

Also, you can press Enter to reset the generation to a fresh starting point, so it should be "playable" again.

We think of our repo as a way to share our research with the community. We did not design it as a fully-featured, playable, user-facing product.

bailehang commented 3 weeks ago

Thank you for your reply and sharing.