Open mengqiDyangge opened 2 years ago
Thank you for your proposal.This method is not planned in our development plan, but we also think the evaluation metrics are very important for the long-tail problem. We welcome you to implement this method, and we are very willing to cooperate with you to further improve the code.
Describe the feature Implement new long-tail dataset evaluation metrics "APpool" and "APfix" in paper "Evaluating Large-Vocabulary Object Detectors: The Devil is in the Details".
Motivation The new evaluation metrics "APpool" and "APfix" are proposed in this paper for evaluating the performance of different methods on the long-tail datasets like LVIS. We think implement these metrics will effectively promote the study of the long-tail problem.
Related resources You can look at this repo of large-vocab-devil based on Detectron2, which is not supported in mmdetection. https://github.com/achalddave/large-vocab-devil
Additional context It really worths to be implemented. After that different models can be evaluated more comprehensively on long-tail datasets such as LVIS.