pangeo-data / climpred

:earth_americas: Verification of weather and climate forecasts :earth_africa:
https://climpred.readthedocs.io
MIT License
233 stars 48 forks source link

monthly HindcastEnsemble.verify() fails #373

Closed aaronspring closed 4 years ago

aaronspring commented 4 years ago

This below code should actually work. we didnt write proper tests here.

inits in november, monthly leads, assim monthly

from climpred.classes import HindcastEnsemble
import numpy as np
import xarray as xr
from climpred.tutorial import load_dataset

hist = load_dataset('CESM-LE')

hind = load_dataset('CESM-DP-SST')
hind['init']=xr.cftime_range(start='1953-11',freq='12MS',periods=hind.init.size)
hind.lead.attrs['units']='months'
hind['init']

# fake a larger assim with monthly data
assim = load_dataset('FOSI-SST')
assim = assim.sel(time=list(assim.time.values)*12)
assim['time']=xr.cftime_range(start='1953',freq='MS',periods=assim.time.size)
assim['time']

he = HindcastEnsemble(hind)
#he = he.add_uninitialized(hist)
he = he.add_observations(assim, "assim")
he

skill = he.verify()
CoordinateError: A common set of verification dates cannot be found for the initializations and verification data supplied. Change `alignment` to 'same_inits' or 'maximize'.
bradyrx commented 4 years ago

I'll give this a look, thanks!

aaronspring commented 4 years ago

We should definitely add these kind of tests.

bradyrx commented 4 years ago

I just looked into this -- it's not a bug. In your example, initializations are annually on November 1st. So you have 11/1953, 11/1954, 11/1955, and so on. The lead dimension is set to be monthly. So these forecasts verify at lead 1 in 12/1953, 12/1954, ... At lead 2 in 1/1954, 1/1955, ...

Since initializations are annually, there's no overlap to have a common set of verification dates at all leads. See the schematic in https://climpred.readthedocs.io/en/latest/alignment.html. If you draw out a couple lead, couple init mockup of your setup there are no diagonals in common. So this case is only feasible for alignment='same_init' or alignment='maximize' (which works here).

You could do:

from climpred.classes import HindcastEnsemble
import numpy as np
import xarray as xr
from climpred.tutorial import load_dataset

hind = load_dataset('CESM-DP-SST')
# Monthly initializations with monthly lead time
hind['init']=xr.cftime_range(start='1953-11',freq='1MS',periods=hind.init.size)
hind.lead.attrs['units']='months'

assim = load_dataset('FOSI-SST')
assim = assim.sel(time=list(assim.time.values)*12)
assim['time']=xr.cftime_range(start='1953',freq='MS',periods=assim.time.size)

he = HindcastEnsemble(hind)
he = he.add_observations(assim, "assim")

he.verify(alignment='same_verifs')