[Pipeline] ReNoise: Real Image Inversion Through Iterative Noising

clarencechen commented 7 months ago

Model/Pipeline/Scheduler description

Achieving faithful image-to-noise inversion with Denoising Diffusion models remains a challenge, particularly for more recent models trained to generate images with a small number of denoising steps. This work introduces an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. Building on reversing the diffusion sampling process, the method applies multiple fixed-point iterations to estimate the next inversion target at each noise level, and then averages the predictions to empirically increase image reconstruction quality. Furthermore, this method preserves editability through optimizing noise regularization losses in a fashion similar to Pix2PixZero.

Open source status

[ ] The model implementation is available.
[ ] The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

Arxiv PDF: https://arxiv.org/pdf/2403.14602.pdf
Github: https://github.com/garibida/ReNoise-Inversion
First Author: @garibida
Notes: The code is not available yet, but the method should be easy to implement as a new pipeline similar to the existing StableDiffusionPix2PixZeroPipeline based on the pseudocode and discussions in the paper.

satani99 commented 7 months ago

hey @sayakpaul i'd like to work on this. can you assign this to me?

sayakpaul commented 7 months ago

Sure, feel free to start a PR :)

github-actions[bot] commented 6 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

garibida commented 6 months ago

Hi, you can use my code as reference: https://github.com/garibida/ReNoise-Inversion

satani99 commented 6 months ago

thanks @garibida

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

huggingface / diffusers