ayushtewari / DFM

Implementation of "Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision"
https://diffusion-with-forward-models.github.io/
147 stars 18 forks source link

Just predicting noise #18

Open 1ssb opened 4 months ago

1ssb commented 4 months ago

It seems like this model is not very robust to trjectory changes, and can only predict a few trajectories, is there any explanation to these completely losing any meaning of the world?

https://github.com/ayushtewari/DFM/assets/40661648/da21477c-0604-4ecc-8c4f-5c869e8019ca

It seems that it's just predicting the next best frame in the sequence that it has been trained on in terms of the dataset trajectories and does not have the capability to take any other target pose. Any help?