HindcastEnsemble.remove_bias() experimental for NWP leads

[aaronspring]

Describe the bug

• cross_validate=True doesnt work at all, if non-annual leads
• unsure about the behaviour in general for non-annual leads
• likely some completely different approach to bias reduction like smoothing needs to be implemented

became apparent in #603

Code Sample

requires #603 #529

with climpred.set_options(seasonality="dayofyear"):
        dayofyear_seasonality = hindcast.remove_bias(
            alignment=alignment, cross_validate=cross_validate
        )
with climpred.set_options(seasonality="weekofyear"):
        weekofyear_seasonality = hindcast.remove_bias(
            alignment=alignment, cross_validate=cross_validate
        )
assert not weekofyear_seasonality.get_initialized().identical(
        dayofyear_seasonality.get_initialized()

Expected behavior

• reduce bias with bias only from seasonality, e.g. initialized.init.dt.dayofyear or initialized.init.dt.month
• enable cross_validate=True: dont bias reduce initialization with that years bias

How to approach

1. Write down how bias reduction is supposed to work
2. Implement code

Easier to implement once

575 is merged: bias is calculated in time but named init

[aaronspring]

Whats the standard in S2S to do bias correction? @judithberner I would appreciate papers well describing the technical part or even some code.

[judithberner]

So the model drifts away from the initialization (e.g. warm bias). So you have to compute the lead-time dependent bias and remove it. IMO - It really masks an important manifestation of model error.

[judithberner]

Here is some code based on Riley's and updated by Abby. My feeling is it's not the latest version .... https://github.com/judithberner/climpred_CESM1_S2S/blob/main/0.03_generate_member_anomalies_S2S.ipynb @abjaye

[aaronspring]

So groupby(init.groupby) - clim. I think that’s what currently remove_bias does

[aaronspring]

What about remove the diurnal cycle?

[judithberner]

So I have very rarely worked with sub-daily data and if so only with 6-hourly precip. Presumably people would want the daily cycle removed, but not always. Maybe something to ask Pauline or someone like her.

[aaronspring]

645 really improved bias reduction

[aaronspring]

Closed by #645