search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
25.18k stars 5.21k forks source link

The implementation of a Null-Text Inversion Pipeline #6313

Closed Junsheng121 closed 7 months ago

Junsheng121 commented 8 months ago

Model/Pipeline/Scheduler description

NullTextInversionPipeline

A pipeline implementation of Null-text Inversion for Editing Real Images using Guided Diffusion Models. As I did not find it in Diffusers, I'd like to add this feature to Diffusers pipeline. The NullTextInversionPipeline allows users to conduct DDIM-Inversion, Null-text Optimization, and DDIM reconstruction via w, w/o Null-text Optimization. No prompt-to-prompt code is implemented as there is a Prompt2PromptPipeline. https://github.com/huggingface/diffusers/blob/2d43094ffc9b1ee377651c6c8a358c81f0c96005/examples/community/pipeline_prompt2prompt.py#L44

Open source status

Provide useful links for the implementation

The official implementation from Google is https://github.com/google/prompt-to-prompt/blob/main/null_text_w_ptp.ipynb Paper: Null-text Inversion for Editing Real Images using Guided Diffusion Models http://openaccess.thecvf.com/content/CVPR2023/html/Mokady_NULL-Text_Inversion_for_Editing_Real_Images_Using_Guided_Diffusion_Models_CVPR_2023_paper.html Null-text Inversion requires no additional model weights. The users may use personal photos to conduct the inversion process. Null-text inversion enables intuitive text-based editing of real images with the Stable Diffusion model. We use an initial DDIM inversion as an anchor for our optimization which only tunes the null-text embedding used in classifier-free guidance. @amirhertz @dmarx @johnnypeck @mbrukman @jikkuatwork

sayakpaul commented 8 months ago

Feel free to add that as a community pipeline.

Junsheng121 commented 8 months ago

Thanks. I will make a PR tomorrow.

Sayak Paul @.***> 于2023年12月25日周一 23:50写道：

Feel free to add that as a community pipeline.

— Reply to this email directly, view it on GitHub https://github.com/huggingface/diffusers/issues/6313#issuecomment-1869034575, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYSXVSFSDK2PHCZX3ASDIILYLGOFJAVCNFSM6AAAAABBCAC37GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRZGAZTINJXGU . You are receiving this because you authored the thread.Message ID: @.***>

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

sayakpaul commented 7 months ago

@Junsheng121 checking if you're still interested to give this a shot?

Junsheng121 commented 7 months ago

Already solved in https://github.com/huggingface/diffusers/commit/d184291c7d2284281269e1257250aaea526ef556. Thanks!

sayakpaul commented 7 months ago

Ah sorry.

PR https://github.com/huggingface/diffusers/pull/6329