Better percentile plots

[gmaze]

Determine why and how to improve figures of percentiles with lower levels cuts or weird values:

[gmaze]

Explanation

Percentiles are computed using profile labels that exists whatever the local profile depth, as long as it was used by the pcm.

The most classic case is when a pcm is based on a shallow part of profiles, which results in the percentile computation to use profile of different absolute depths.

This results in percentiles by depth that are based on samples of very different size and possibly from very different structures.

This simply means that correlations found by the classifier over the PCM layer are not relevant for the bottom layers.

I think this is why we have this results in percentile plots.

[gmaze]

This issue is more general that BlueCloud requirements and should be transferred to the pyxpcm repo

[gmaze]

Reproducing the issue

# Load data
from pyxpcm.models import pcm
ds = pyxpcm.tutorial.open_dataset('isas_series').load()

# Remove low latitude (<41N) and deep values (<1000m) on purpose to simulate dramatic changes in topography
def rem(da):
    t = da.values
    t[:,100::,0:30,:] = np.nan
    da.values = t
    return da
ds['TEMP'] = rem(ds['TEMP'])
ds['PSAL'] = rem(ds['PSAL'])

# PCM analysis
z = ds['depth'][0:20].values
pcm_features = {'temperature': z}
m = pcm(K=8, features=pcm_features)

features_in_ds = {'temperature': 'TEMP'}
features_zdim='depth'
m.fit_predict(ds, features=features_in_ds, dim=features_zdim, inplace=True)

# Compute quantiles
for vname in ['TEMP']:
    ds = ds.pyxpcm.quantile(m, q=[0.05, 0.5, 0.95], of=vname, outname=vname + '_Q', keep_attrs=True, inplace=True)

# Plot quantiles
fig, ax = m.plot.quantile(ds['TEMP_Q'], maxcols=4, figsize=(10, 8), sharey=True)

We clearly see the break in curves at 1000m, which is the depth where we arbitrary modified values to simulate a dramatic changes in topography