NRCan / geo-deep-learning

Deep learning applied to georeferenced datasets
https://geo-deep-learning.readthedocs.io/en/latest/
MIT License
150 stars 49 forks source link

"[BUG]: NaN metric values" #569

Open FatemehEsfahani opened 2 months ago

FatemehEsfahani commented 2 months ago

Describe the bug When a class does not show up in a batch, the corresponding row in the confusing matrix becomes zero. As a result, division by zero happens and performance metrics for that class becomes NaN. https://github.com/NRCan/geo-deep-learning/blob/8c8250c2184606919be7c086e56e51a82df6cd39/utils/metrics.py#L130

To Reproduce Steps to reproduce the behavior:

  1. Test with an image whose label has 3 classes including background class (i.e. class_0, class_1, class_2) in which only class_0 and class_1 are available.

Expected behavior The metric calculation should handle division by zero and set a value (i.e., 0) when a class is absent.

Screenshots Example of the confusion matrix for a batch: image Then, the metrics for class_2, based on the following formula becomes NaN: image

Additional context Small epsilon value can be added in the denominator of each metric calculation formua to avoid devision by zero. Something like the following: image