search

NOAA-OWP / gval

A high-level Python framework to evaluate the skill of geospatial datasets by comparing candidates to benchmark maps producing agreement maps and metrics.
https://noaa-owp.github.io/gval/
Other
22 stars 0 forks source link

Assigning FNs when `negative_categories=None` within `compute_metrics_table()` #107

Closed fernando-aristizabal closed 1 year ago

fernando-aristizabal commented 1 year ago

FN's are currently being assigned when negative_categories=None within compute_metrics_table()

Current behavior

Using the following crosstab table:

  cdf_multi = pd.DataFrame(
      {
          "band": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
          "candidate_values": [1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4],
          "benchmark_values": [1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4],
          "counts": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
      }
  )

and setting negative_categories=None:

    metrics_df = _compute_categorical_metrics(
        cdf_multi, metrics="critical_success_index",
        positive_categories=[1,2,3],
        negative_categories=None,
        average="weighted",
        weights=[1, 2, 3]
    )

Crosstab_df with fn conditions being assigned erroneously:

   band  candidate_values  benchmark_values  counts  weights conditions                                                                                           
0     1               1.0               1.0     1.0      1.0         tp                                                                                           
1     1               1.0               2.0     2.0      1.0         tp                                                                                           
2     1               1.0               3.0     3.0      1.0         tp                                                                                           
3     1               2.0               4.0     4.0      2.0         fp                                                                                           
4     1               2.0               1.0     5.0      2.0         tp
5     1               2.0               2.0     6.0      2.0         tp
6     1               3.0               3.0     7.0      3.0         tp
7     1               3.0               4.0     8.0      3.0         fp
8     1               3.0               1.0     9.0      3.0         tp
9     1               4.0               2.0    10.0      NaN         fn
10    1               4.0               3.0    11.0      NaN         fn
11    1               4.0               4.0    12.0      NaN       None

Even though functionality not currently in main is illustrated, the behavior is still there.

Expected behavior

FNs should be assigned when negative_categories=None