search

Junyi42 / sd-dino

Official Implementation of paper "A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence"
https://sd-complements-dino.github.io
271 stars 14 forks source link

Model parameter mismatch #16

Closed XLechter closed 6 months ago

XLechter commented 6 months ago

Hi, thanks for sharing the codes.

I found a problem when running the demo codes. I followed all the setup in readme without changing anything, but it seems the download pre-trained weights mismatch the model:

image

so I got the results which are very different from yours: image

This problem also occurs when I run Geoaware-SC. Could you give me some advice on how to solve this?

Junyi42 commented 6 months ago

Hi, thanks for your interest in our work!

The problem of weights mismatch is as expected, because the features we used for correspondence is different from the one used to generate the segmentation mask in ODISE (more details in Supp. Tab. 2). One simple fix is to set decoder_only=False for generating the mask, though the correspondence quality might be slightly lower.

Please refer to #12 and #1 (both are similar or same issue) for more details. Thanks!

XLechter commented 6 months ago

@Junyi42 Thank you so much. My issue is well resolved.