search

Haiyang-W / GiT

[ECCV2024 Oral🔥] Official Implementation of "GiT: Towards Generalist Vision Transformer through Universal Language Interface"
https://arxiv.org/abs/2403.09394
Apache License 2.0
303 stars 13 forks source link

Mismatch in few-shot and zero-shot numbers between paper and code #12

Closed NareshGuru77 closed 2 months ago

NareshGuru77 commented 2 months ago

There are discrepancies in the results of few-shot results and in one zero shot result as shown below:

git_discrepancies

However, results for zero-shot cs-det, cs-seg and sun - rgbd seg match with only slight deviations.

Would it be possible that I am looking at the wrong metric or maybe there is some other issue? Would be glad if you could help me with this. Thank you!.

nnnth commented 2 months ago

I think your metric is correct. We retrained the model after the paper was published, so the performance may differ from what was reported. Based on my experience, few-shot performance can vary significantly.

NareshGuru77 commented 2 months ago

Thank you for your quick response!. So I can use the current numbers I get as the new results ?

Haiyang-W commented 2 months ago

yeah, I think so. :) Enjoy with the code.

NareshGuru77 commented 2 months ago

Thank you!