eric-ai-lab / photoswap

Official implementation of the NeurIPS 2023 paper "Photoswap: Personalized Subject Swapping in Images"
https://photoswap.github.io
MIT License
342 stars 24 forks source link

Poor performance on custom instance #11

Open zhangn77 opened 1 year ago

zhangn77 commented 1 year ago

Thank you for your work, the performance of the provided demo is excellent. I follow your instruction and train my own concept model by DreamBooth with text-encoder fine-tuned. Through validation, it can generate some good imgs itself. However, when I use it as the concept model and run the swap func, I cannot get meaningful results, even in any combinations of [cross_map_replace_steps, self_output_replace_steps, self_map_replace_steps]. I wonder which part is wrong? I think the step1 may have some mistakes, because when I change it to the provided taylor, the results are good with the same source img, but the concept model I trained can already generate the target concept.

g-jing commented 11 months ago

Sorry for the late response. Could you provide more details? For example, what diffusers version did you use? Do you mean you can not generate good results with your concept model and your source image, or your concept model and the provided Prof. Feifei Li image?

Also, one tip is to use around 20 images to train the DreamBooth, which might increase the performance.