search

iri-pycpt / pycpt

package joining pycpt stuff
3 stars 1 forks source link

Implement skill masking #63

Open awrobertson opened 5 months ago

awrobertson commented 5 months ago

I will add to cca.py etc
if skillmask:
cpt.write(5372) ...

What does CPT expect after that. Should a real value (eg 0.) then be passed?

However, can skill masking be applied when using cross-validated hindcasts, or only when using the retroactive option? PyCPT seasonal does not use the latter, so skill masking might not be possible.

CPT help says: https://cpthelp.iri.columbia.edu/Options_Masking.html

Skill-masking applies only when calculating retroactive forecasts and when running Tools ~ Forecast . If the skill is less than the threshold indicated by "skill mask" then climatological forecasts will be issued. The skill can be measured using: Pearson's correlation Spearman's correlation Kendall's tau Kendall's tau-c The skill is calculated using either the cross-validated hindcasts, or retroactive hindcasts, or fitted values depending upon how the forecast error variance (as set in Options ~ Forecast ) is calculated. The skill masking can be reset after running Actions ~ Calculate .

awrobertson commented 5 months ago
PyCPT Skill masking

Added to pcr.py and cca.py

And

if skillmask:   
    cpt.write(5372)
    cpt.write('Y')
    cpt.write(1)    # for Pearson
    cpt.write(skillmask_value)

plus bookkeeping to configuration.py

def check_cpt_args(args): args = copy.copy(args) keys = ['transform_predictand', 'tailoring', 'cca_modes', 'x_eof_modes', 'y_eof_modes', 'crossvalidation_window', 'synchronous_predictors', 'scree', 'drymask', 'drymask_value','skillmask','skillmask_value','validation'] # AWR 04/06/24

keytypes = {'transform_predictand': str, 'tailoring': str, 'cca_modes': tuple, 'x_eof_modes': tuple, 'y_eof_modes': tuple, 'crossvalidation_window': int, 'synchronous_predictors': bool, **<img width="1887" alt="PyCPT Skill masking" src="https://github.com/iri-pycpt/pycpt/assets/21113530/318268bf-0776-4c7c-a3d5-c305efcf9d63">** 'scree': bool, 'validation': str}