search

rafaelpadilla / review_object_detection_metrics

Object Detection Metrics. 14 object detection metrics: mean Average Precision (mAP), Average Recall (AR), Spatio-Temporal Tube Average Precision (STT-AP). This project supports different bounding box formats as in COCO, PASCAL, Imagenet, etc.
Other
1.09k stars 215 forks source link

Command Line Interface #57

Closed vjsrinivas closed 2 years ago

vjsrinivas commented 3 years ago

As requested, here is the first implementation of #25 with all the core features from the GUI tool. I've adjusted my test_cli.py to utilize some of the tests done in the respective COCO and VOC tests, but I'm open to expanding the tests to make sure everything is truly kosher.

vjsrinivas commented 3 years ago

Question: Is the plot_coco_pr_graph function in cli.py redundant? I know there is a PR plotting function for Pascal VOC, but I could not find a similar one for COCO.

If there isn't one for COCO, I suggest moving the graphing function to coco_evaluator.py. If it does exist, please point me to it so I can replace my redundant code.

rafaelpadilla commented 3 years ago

Question: Is the plot_coco_pr_graph function in cli.py redundant? I know there is a PR plotting function for Pascal VOC, but I could not find a similar one for COCO.

If there isn't one for COCO, I suggest moving the graphing function to coco_evaluator.py. If it does exist, please point me to it so I can replace my redundant code.

Currently, the AP plotting is made considering only Pascal VOC. While COCO only provides numerical metrics, Pascal VOC provides numerical metrics and the plotting.

As only Pascal VOC official tool provides plottings, for a sake of tools comparison, I think it is better to leave things how they are.

vjsrinivas commented 3 years ago

Question: Is the plot_coco_pr_graph function in cli.py redundant? I know there is a PR plotting function for Pascal VOC, but I could not find a similar one for COCO. If there isn't one for COCO, I suggest moving the graphing function to coco_evaluator.py. If it does exist, please point me to it so I can replace my redundant code.

Currently, the AP plotting is made considering only Pascal VOC. While COCO only provides numerical metrics, Pascal VOC provides numerical metrics and the plotting.

As only Pascal VOC official tool provides plottings, for a sake of tools comparison, I think it is better to leave things how they are.

Just clarifying, do you want the PR graphing for COCO disabled all together then? We can throw an exception if the user attempts it.

vjsrinivas commented 3 years ago

Hi @rafaelpadilla, I'm sorry about the lack of response. I got very busy with work, but I'll try to look over your reviews this weekend.

vjsrinivas commented 3 years ago

@rafaelpadilla Sorry for the delay! Let me know if there are any other modifications needed.

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Xu-Justin commented 2 years ago

Still waiting for this to be approved :smile: I need a CLI version to automate my benchmarking system.

vjsrinivas commented 2 years ago

@Xu-Justin Unfortunately, I'm very busy with my studies and haven't gotten feedback since the last branch commit. You can take a look at my branch and take the cli.py. I'm not sure if it still works with the latest version of the repo, but it is worth a shot.