search

google-deepmind / magiclens

[ICML'24 Oral] "MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions"
Apache License 2.0
132 stars 8 forks source link

DTIN evaluation code #11

Open JUNJIE99 opened 1 month ago

JUNJIE99 commented 1 month ago

Hi,

I am interested in reproducing the results from the DTIN dataset. Could you please share the evaluation code for DTIN?

Thank you.

drogozhang commented 1 month ago

Hi, thanks for your interest. We haven't written the DTIN evaluation code for current implementation. It is just models transferred from inside parameters after testing on CIRCO and FashionIQ.

I will let you know when I have the bandwidth to implement DTIN here.

drogozhang commented 2 weeks ago

Hi, I've updated the code on https://github.com/drogozhang/magiclens and working on a PR! Let me know if you have any other questions.

drogozhang commented 2 weeks ago
image

Although performance drops a bit due to the model weight conversion it is still the best results on DTIN benchmark iirc. Also, you can try different prompts in data_tuils.py for better results (I didn’t heavily explored the prompts before).