search

zhmiao / OpenLongTailRecognition-OLTR

Pytorch implementation for "Large-Scale Long-Tailed Recognition in an Open World" (CVPR 2019 ORAL)
BSD 3-Clause "New" or "Revised" License
839 stars 128 forks source link

How to get openset accuracy? #48

Closed Qoo95 closed 4 years ago

Qoo95 commented 4 years ago

Thanks for sharing your work. May I ask how to calculate openset accuracy? Because, before calculating openset accuracy, it has 

probs, preds = F.softmax(self.total_logits.detach(), dim=1).max(dim=1)

so that open_threshold doesn't work. https://github.com/zhmiao/OpenLongTailRecognition-OLTR/blob/fb6203c8e1cfa7ed5bf9b1acf581800d04f92c4a/run_networks.py#L296

zhmiao commented 4 years ago

Hello @Qoo95 , the openset accuracy is calculated here: https://github.com/zhmiao/OpenLongTailRecognition-OLTR/blob/master/run_networks.py#L297

Open threashold won't be used if main argument test_open is not True.

Does that make sense?

zhmiao commented 4 years ago

It has been 26 days since last comment. I will close this issue now. If you have any more questions, please reopen this issue. Thanks.

twmht commented 3 years ago

@zhmiao

so if probs < self.training_opt['open_threshold'] is equal to true, this input image is regard as an open set input (novel class)?