search

ktindiana / sphinxval

SPHINX validation code for solar energetic particle models
MIT License
3 stars 3 forks source link

Unit tests and documentation for metrics #56

Open rickyegeland opened 5 months ago

rickyegeland commented 5 months ago

Develop a suite of unit tests to verify that all of the formulae are correct, and that the function call works. These should be based on simple contingency tables, e.g. [[1, 1], [1, 1]], or continuous datasets such that the formula result can easily be verified by hand with pencil and paper.

Coverage should include all metrics in metrics.py.

Furthermore, code documentation should be improved to include a reference where each metric definition can be found. Journal articles or books are preferred to web sites. Comments should be used to give alternative names for each metric in the code, e.g. 

    'H': check_div(h, h+m),  # Hit Rate, Probability of Detection, Recall
    'FAR': check_div(f, h+f),  # False Alarm Ratio

Names used in the reporting tool or found in relevant resources should be used. Some quantities have several names.

rickyegeland commented 5 months ago

@claya71 I think this one would best go to you given your recent work.

rickyegeland commented 4 months ago

@claya71 can you write a comment here so I can assign this to you? GitHub is not allowing it right now.

claya71 commented 4 months ago

A comment

rickyegeland commented 2 months ago

Waiting for review of documentation of metrics in references subdirectory. @lukestegeman @ktindiana or @rickyegeland can help.