search

keras-team / keras

Deep Learning for humans
http://keras.io/
Apache License 2.0
61.34k stars 19.39k forks source link

F2 Score and mAP metric for object detection model. #15811

Closed pure-rgb closed 1 year ago

pure-rgb commented 2 years ago

I didn't find a metric regarding the object detection task in keras.

How to implement the above two metrics in keras and use them? In this competition, https://www.kaggle.com/c/tensorflow-great-barrier-reef/overview/evaluation it uses F2 score for its object detection task but I don't know how to evaluate my model locally with this score?

The metric sweeps over IoU thresholds in the range of 0.3 to 0.8 with a step size of 0.05, calculating an F2 score at each threshold. For example, at a threshold of 0.5, a predicted object is considered a "hit" if its IoU with a ground truth object is at least 0.5.

chunduriv commented 2 years ago

@Suzan009,

Can you check below metrics and let us know if it helps you?

F2 Score : tfa.metrics.FBetaScore withbeta=2.0

mAP: tfr.keras.metrics.MeanAveragePrecisionMetric

Thanks!

pure-rgb commented 2 years ago

@chunduriv

actually no. check this, tfa.metrics.FBetaScore, it clearly states and it's not suitable for object detection. And same as this one too tfr.keras.metrics.MeanAveragePrecisionMetric

Please correct me with a working example if I miss something.

sachinprasadhs commented 2 years ago

You can find the details in the supported object detection evaluation protocols here.

pure-rgb commented 2 years ago

Please note that I need keras implementation so that I can integrate it conveniently into my pipeline. That repo is a complete mess for me. I also loss work motivation when I see that repo, no offense.

google-ml-butler[bot] commented 2 years ago

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

google-ml-butler[bot] commented 2 years ago

Closing as stale. Please reopen if you'd like to work on this further.

sachinprasadhs commented 1 year ago

@pure-rgb , Keras implementation of metric FBetaScore will be available from TensorFlow 2.13 version, here is the document link for the detail. For now, you can make use of this using tf-nightly version. Let us know if this addresses your concern. Thanks!

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.