The new class PerformanceTracker in performance_metrics measures how good the master equation model is for class predictions. By default it tracks the unobserved I and E statuses using a threshold level of 50% for the master equations probabilities using the two metrics below.
Two metrics are implemented.
Model accuracy: simply computes the number of correct classifications. Any good model in the presence of balanced data (uniformly observed classes) is required to have an accuracy better than random. For unbalanced classes this metric (in blue) should be compared against the prevalence level (in red).
F1 score: is a performance metric for binary classification tasks, specially useful for unbalanced datasets. A perfect F1 score of 1.0 is only achievable with perfect recall and precision.
More in the wiki
An accompanying example file can be found in /examples/simple_epidemic_with_da_health_and_death_records.py. Which generates a figure as below.
MWE:
model_performance = PerformanceTracker([list of metrics], statuses = ['E', 'I'], threshold = .50)
:
# simulation code
:
model_performance.update(epidemic_simulator.kinetic_model.current_statuses, states_ensemble)
# prints the current state of the metrics.
model_performance.print()
:
The new class
PerformanceTrackerinperformance_metricsmeasures how good the master equation model is for class predictions. By default it tracks the unobservedIandEstatuses using a threshold level of 50% for the master equations probabilities using the two metrics below.Two metrics are implemented.
Model accuracy: simply computes the number of correct classifications. Any good model in the presence of balanced data (uniformly observed classes) is required to have an accuracy better than random. For unbalanced classes this metric (in blue) should be compared against the prevalence level (in red).F1 score: is a performance metric for binary classification tasks, specially useful for unbalanced datasets. A perfect F1 score of1.0is only achievable with perfectrecallandprecision. More in the wikiAn accompanying example file can be found in
/examples/simple_epidemic_with_da_health_and_death_records.py. Which generates a figure as below.MWE: