search

Yifei-Y / Openset-RCNN

This repository contains the code for the IEEE Robotics and Automation Letters paper "Open-Set Object Detection Using Classification-Free Object Proposal and Instance-Level Contrastive Learning"
Apache License 2.0
12 stars 1 forks source link

New AOSE Per class #4

Closed jannehlamin closed 4 months ago

jannehlamin commented 4 months ago

Sorry for engaging you again, I want to add another evaluation metric in addition "AOSE" but your logic is a bit complicated.

p

In the evaluation, the unknown false positive (ufps) capture 'unknown class' classified as any of the 'known classes'. For instance, known classes are kc=[1, 2,3, 4, 5] and unknown class is ukc=[1000] Therefore, whenever unknown class is misclassified as any of the known classes, it will be added to the ufps alarm. However, I want to consider a different situation i.e whenever unknown class is misclassified as kc[0], i.e know class at index 0 then it should be added to the ufps.

ufps = np.logical_and(kdtm_unkgt, np.logical_not(kdtIg_unkgt)) ufps_per_target_class = np.logical_and(kdtm_unkgt, np.logical_not(kdtIg_unkgt)) # logic ? using your logic, how do I implement that on this line : ufps_per_target_class.

jannehlamin commented 4 months ago

I have been working through your unofficial implementation and extracted the target class information successfullly as follows:

  1. get the class detection and ground truth information

    • target_gts = self.cocoGt.loadAnns(self.cocoGt.getAnnIds(imgIds=p.imgIds, catIds=targetId))
    • target_dts = self.cocoDt.loadAnns(self.cocoDt.getAnnIds(imgIds=p.imgIds, catIds=targetId))

  2. after update other information, we finally get here but the logic for target_fps is wrong .

how to get this logic?

jannehlamin commented 4 months ago

After prolong analysis, I have come to the conclusion that the target false positive count is enough to get TargetId of a class that is misclassified as any other class. This is a new metrics I need for my case.Thanks!

Yifei-Y commented 4 months ago

I apologize for the delay in responding to your issue. I'm glad to hear that you have found a solution.