search

WindVChen / Diff-Harmonization

A novel zero-shot image harmonization method based on Diffusion Model Prior.
Apache License 2.0
110 stars 8 forks source link

Wrong Result？ #8

Closed StarsTesla closed 1 month ago

StarsTesla commented 1 month ago

Hi, I follow the readme, install and run the demo, then I get like 9 output, as shows in image, and the girl is like get worse face? image

WindVChen commented 1 month ago

Hi @StarsTesla ,

I guess "the worse face" you're referring to is related to the structure of the foreground image (the girl), where there is distortion in the nose and blurring around the right eye (from the observer's perspective). This distortion can occur due to the stochastic nature of the diffusion generative model and the zero-shot approach of our method, which does not involve fine-tuning the generative model. We also discuss this limitation in Section 5 of our paper.

Since we haven’t yet identified an optimal way to fully preserve the foreground structure, a temporary solution could be to rerun the algorithm and see if the structure improves, taking advantage of the model's stochasticity. Alternatively, you might try increasing the loss factor, such as the edge constraint loss, though this could reduce the intensity of the harmonization.

Hope this helps.