search

sgvaze / osr_closed_set_all_you_need

Official repo for our ICLR 22 Oral paper: "Open-Set Recognition: a Good Closed-Set Classifier is All You Need?"
MIT License
267 stars 32 forks source link

How to predict a single img with label known or unknown? #15

Closed LP308210365 closed 2 years ago

LP308210365 commented 2 years ago

Hello again! I'm working on a task with your code, i have seen that all methods are evaluate by AUROC, a threshold independent metric, but now i need to predict the label by threshold, is there any func in code to do that?

LP308210365 commented 2 years ago

I found the func to get the threshold:"test/utils/find_nearest"

sgvaze commented 2 years ago

Hi! test.utils.find_nearest.acc_at_t might be what you're looking for. It computes binary (open-set/closet-set) accuracy at a given threshold.

The choice of threshold will depend on your downstream application (e.g do you care about false positives or false negatives) and would typically be chosen on some validation examples. We didn't tackle this here! Hope this helps.

LP308210365 commented 2 years ago

thanks a lot, this is what i want, i train the classifier32 with custom dataset, i think the given threshold can work in this domain