search

google / prompt-to-prompt

Apache License 2.0
2.98k stars 279 forks source link

Error introduced when using p2p pipeline comparing to null-text inversion #86

Open KolvacS-W opened 3 months ago

KolvacS-W commented 3 months ago

Hey, I encounter a weird issue that when I use p2p, the reconstruction of the original image (which is successfully reconstructed before by null-text inversion) has errors:

My original and reconstructed image by inversion:

Screen Shot 2024-04-01 at 4 11 43 PM

outputs when using p2p:

Screen Shot 2024-04-01 at 4 15 55 PM

Even when I use the origi

Screen Shot 2024-04-01 at 4 05 49 PM

nal controller without any attention swap, the error exists as long as I have a new prompt:

Screen Shot 2024-04-01 at 3 47 47 PM

Seems like when a second prompt is added, it will affect the context parameter for ptp_utils.diffusion_step(model, controller, latents, context, t, guidance_scale, low_resource=False), which affects the prediction: noise_pred = model.unet(latents_input, t, encoder_hidden_states=context)["sample"] in p2p_utils.py.

I am wondering if anybody know why I have this issue? Thanks! (image credit source: https://billf.mit.edu/about/shapetime)

Dijkstra14 commented 3 months ago

I've encountered the same issue. Anyone know the reason for this?