chenhonghua
commented
3 months ago I see.
We do not have any further processing.
Yes, during my initial tests, I also observed that the results of SDS were not stable. I found that the timing schedule is crucial for achieving stable results, although the depth was somewhat lacking. Therefore, I experimented with both depth SDS and normal SDS. We discovered that normal SDS could recover the geometry to a certain extent, which helps enhance the RGB SDS optimization.
Regarding the consistency issue, I believe SDS optimization is more effective than directly using 2D inpainting results (from Lama or the SD inpainting model). However, if we can achieve consistent 2D inpainting results, I believe, this kind of consistent pixel supervision will be better than SDS optimization.
Arcxml
Hello, thank you very much for your work. I would like to ask a question. In your work, is the mask area only supervised by two SDSs, RGB and geometry, and there is no pixel supervision like the image generated by diffusion?
I have previously conducted experiments similar to using only SDS RGB for supervision, and found that the effect in the mask area is not so good.
And according to my experiments, the diffusion model cannot generate such consistent images.
Is there any other processing? Looking forward to your reply.