search

lucidrains / recurrent-interface-network-pytorch

Implementation of Recurrent Interface Network (RIN), for highly efficient generation of images and video without cascading networks, in Pytorch
MIT License
194 stars 14 forks source link

Potential Issue with Self-Conditioning #8

Closed justinlovelace closed 1 year ago

justinlovelace commented 1 year ago

Hello Phil! Thank you for all of your excellent open-source implementations!

I noticed a potential bug with self-conditioning in pixel-space. If the model is trained to predict eps instead of x_0, then I think the model is currently being conditioned on the predicted epsilon instead of the predicted image.

I shared the relevant snippet of code below. It looks like it's conditioned on the output of the network regardless of whether it's being trained to predict x_0 or eps.

My apologies if I'm misunderstanding something!

image
lucidrains commented 1 year ago

@justinlovelace oh yes :facepalm:

i added the epsilon objective at some point but forgot to account for that

thank you Justin! can you see if this commit looks good?

justinlovelace commented 1 year ago

Looks good to me! Appreciate the quick response!