cure-lab / PnPInversion

[ICLR2024] Official repo for paper "PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code"
212 stars 5 forks source link

The single image edit code does not seem to edit the round cake to square. #2

Closed RahulSajnani closed 8 months ago

RahulSajnani commented 8 months ago

Hi

When I run the single image directinversion+p2p edit code it does not change the round cake to a square one. It gives the below output. Also, what are blend words exactly? Can you give me an idea of what words to blend so that I can use custom images and prompts?

directinversion+p2p

juxuan27 commented 8 months ago

Hi, @RahulSajnani. Thank you for your focus. Please check if you have used the correct diffuser version as listed in readme.

RahulSajnani commented 8 months ago

Hi @juxuan27! Thanks for getting back. I ran the single image p2p script and my environment is built using the p2p_requirements.txt file. My diffusers version is 0.10.0

juxuan27 commented 8 months ago

The blended word is the target object that needed to be edit, you could use "cake cake" in this example (as we have given in .ipynb). Would you please give me more detailed usage in your code so that maybe I could find why this happen?

RahulSajnani commented 8 months ago

Hi @juxuan27. Your cake example is not working for me (single image p2p). I am using the diffusers 0.10.0

juxuan27 commented 8 months ago

Yes, I understand. But would you please give me details on how you run it? For example, the exact python file and the corresponding running scripts such as python xxx.py xxx.

RahulSajnani commented 8 months ago

Hi @juxuan27. Thanks for getting back. I run it as follows:

python -u run_editing_p2p_one_image.py --image_path scripts/example_cake.jpg --original_prompt "a round cake with orange frosting on a wooden plate" --editing_prompt "a square cake with orange frosting on a wooden plate" --blended_word "cake cake" --output_path "directinversion+p2p.jpg" "ddim+p2p.jpg" --edit_method_list "directinversion+p2p" "ddim+p2p"

I followed the following instructions for creating my conda environment:

conda create -n p2p python=3.9 -y
conda activate p2p
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
pip3 install -r environment/p2p_requirements.txt
juxuan27 commented 8 months ago

Sorry for the late reply. It's quite weird... I run the same command as you listed and get correct editing results. Would you please try "null-text-inversion+p2p" and see if it can work? If not, I think there is something wrong with your local environment.

zhaocong1992 commented 8 months ago

same probelm.

juxuan27 commented 8 months ago

Hi, @zhaocong1992 ! Would you please try this and give me some feedback on this issue? Hope we could solve this problem together because everything works fine on my cluster and I really don't know what's going on hhh.

ezgiekiz commented 8 months ago

I followed the same setup listed above and run_editing_p2p_one_image.py gives the unedited results as @RahulSajnani shared. However, running run_editing_p2p_one_image.ipynb gives the correct results. You may consider to check the difference between them. Thank you.

juxuan27 commented 8 months ago

Hi, @ezgiekiz ! Thank you for your help. I will check the reason causing this problem and fix the bug.

juxuan27 commented 8 months ago

The bug is fixed in this commit. Thank you for all your help!!