search

ksaito-ut / openworld_ldet

MIT License
54 stars 2 forks source link

What is the evaluation code bug? #4

Closed wjn922 closed 2 years ago

wjn922 commented 2 years ago

Hi,

I notice that you update the README and said there is an evaluation code bug. Could you plz give more information about this?

ksaito-ut commented 2 years ago

Hi, I have updated the evaluation code. The pascal_ids were not indexed correctly. The problem is fixed now. But, we made another little update in the evaluation protocol in our new paper. So, we will further update the evaluation part soon.

wjn922 commented 2 years ago

Well, that would be a problem. Then I suppose the performance would be reduced. Could you plz share the updated paper?Many thanks.

ksaito-ut commented 2 years ago

Hi, sorry for the late reply. We have uploaded paper and available at https://github.com/ksaito-ut/openworld_ldet/blob/main/docs/paper.pdf.

wjn922 commented 2 years ago

Thanks.

wjn922 commented 2 years ago

Hi,

I run your evaluation code and found that there are two metrics. For example, in the figure below (voc->non-voc setting), the AR of classwise is slightly higher than the previous one. Could you please help me tell the differences? Thanks! image

wjn922 commented 2 years ago

By the way, I also wonder how to evaluate the ALL metrics shown in Table 1 and Table 8?