Ocean Color - Restarting Kernel

[chiaravol]

Notebook description

Hi, I need to download and spatially average ocean color data stored in NetCDF files. Everything seemed to work fine, but when performing a spatial weighted mean over a longer dataset I have this error message "The kernel for Ocean_color_v6.ipynb appears to have died. It will restart automatically".

Notebook link or upload

http://localhost:5678/lab/tree/Ocean_color_v6.ipynb

Anything else we need to know?

I had to stringify netCDF files dates when updating the request.

Environment

<

affine 2.4.0 pyhd8ed1ab_0 conda-forge aiofiles 22.1.0 py310hecd8cb5_0
aiosqlite 0.18.0 py310hecd8cb5_0
anyio 3.5.0 py310hecd8cb5_0
appnope 0.1.2 py310hecd8cb5_1001
argon2-cffi 21.3.0 pyhd3eb1b0_0
argon2-cffi-bindings 21.2.0 py310hca72f7f_0
asttokens 2.0.5 pyhd3eb1b0_0
attrs 22.1.0 py310hecd8cb5_0
babel 2.11.0 py310hecd8cb5_0
backcall 0.2.0 pyhd3eb1b0_0
beautifulsoup4 4.12.2 py310hecd8cb5_0
bleach 4.1.0 pyhd3eb1b0_0
blosc 1.21.4 heccf04b_0 conda-forge boltons 23.0.0 py310hecd8cb5_0
boost-cpp 1.78.0 hf5ba120_3 conda-forge brotli 1.0.9 hb7f2c08_9 conda-forge brotli-bin 1.0.9 hb7f2c08_9 conda-forge brotlipy 0.7.0 py310hca72f7f_1002
bzip2 1.0.8 h1de35cc_0
c-ares 1.19.1 h0dc2134_0 conda-forge ca-certificates 2019.11.28 hecc5488_0 conda-forge/label/cf202003 cairo 1.16.0 h09dd18c_1016 conda-forge cdsapi 0.1.6 py_0 conda-forge/label/cf202003 certifi 2023.5.7 py310hecd8cb5_0
cffi 1.15.1 py310h6c40b1e_3
cfitsio 4.2.0 hd56cc12_0 conda-forge cftime 1.6.2 py310h936d966_1 conda-forge charset-normalizer 2.0.4 pyhd3eb1b0_0
click 8.1.3 unix_pyhd8ed1ab_2 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.7.2 pyhd8ed1ab_1 conda-forge comm 0.1.2 py310hecd8cb5_0
conda 23.5.0 py310h2ec42d9_1 conda-forge conda-content-trust 0.1.3 py310hecd8cb5_0
conda-package-handling 2.0.2 py310hecd8cb5_0
conda-package-streaming 0.7.0 py310hecd8cb5_0
contourpy 1.1.0 py310h88cfcbd_0 conda-forge cryptography 38.0.4 py310hdd0c95c_0 conda-forge curl 8.1.2 hbee3ae8_0 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge debugpy 1.5.1 py310he9d5cce_0
decorator 5.1.1 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
entrypoints 0.4 py310hecd8cb5_0
executing 0.8.3 pyhd3eb1b0_0
expat 2.5.0 hf0c8a7f_1 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.2 h5bb23bf_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.40.0 py310h6729b98_0 conda-forge freetype 2.12.1 hd8bbffd_0
freexl 1.0.6 hb7f2c08_1 conda-forge geos 3.12.0 he965462_0 conda-forge geotiff 1.7.1 h5cf5d3c_9 conda-forge gettext 0.21.1 h8a4c099_0 conda-forge giflib 5.2.1 hb7f2c08_3 conda-forge hdf4 4.2.15 h9804679_6 conda-forge hdf5 1.14.1 nompi_hedada53_100 conda-forge icu 72.1 h7336db1_0 conda-forge idna 3.4 py310hecd8cb5_0
ipykernel 6.19.2 py310h20db666_0
ipython 8.12.0 py310hecd8cb5_0
ipython_genutils 0.2.0 pyhd3eb1b0_1
jedi 0.18.1 py310hecd8cb5_1
jinja2 3.1.2 py310hecd8cb5_0
json-c 0.16 h01d06f9_0 conda-forge json5 0.9.6 pyhd3eb1b0_0
jsonpatch 1.32 pyhd3eb1b0_0
jsonpointer 2.1 pyhd3eb1b0_0
jsonschema 4.17.3 py310hecd8cb5_0
jupyter_client 8.1.0 py310hecd8cb5_0
jupyter_core 5.3.0 py310hecd8cb5_0
jupyter_events 0.6.3 py310hecd8cb5_0
jupyter_server 2.5.0 py310hecd8cb5_0
jupyter_server_fileid 0.9.0 py310hecd8cb5_0
jupyter_server_terminals 0.4.4 py310hecd8cb5_1
jupyter_server_ydoc 0.8.0 py310hecd8cb5_1
jupyter_ydoc 0.2.4 py310hecd8cb5_0
jupyterlab 3.6.3 py310hecd8cb5_0
jupyterlab_pygments 0.1.2 py_0
jupyterlab_server 2.22.0 py310hecd8cb5_0
kealib 1.5.1 h7014c1b_4 conda-forge kiwisolver 1.4.4 py310ha23aa8a_1 conda-forge krb5 1.20.1 h049b76e_0 conda-forge lcms2 2.15 h2dcdeff_1 conda-forge lerc 4.0.0 hb486fe8_0 conda-forge libaec 1.0.6 hf0c8a7f_1 conda-forge libarchive 3.6.2 h0b5dc4a_1 conda-forge libblas 3.9.0 17_osx64_openblas conda-forge libbrotlicommon 1.0.9 hb7f2c08_9 conda-forge libbrotlidec 1.0.9 hb7f2c08_9 conda-forge libbrotlienc 1.0.9 hb7f2c08_9 conda-forge libcblas 3.9.0 17_osx64_openblas conda-forge libcurl 8.1.2 hbee3ae8_0 conda-forge libcxx 16.0.6 hd57cbcb_0 conda-forge libdeflate 1.18 hac1461d_0 conda-forge libedit 3.1.20221030 h6c40b1e_0
libev 4.33 haf1e3a3_1 conda-forge libexpat 2.5.0 hf0c8a7f_1 conda-forge libffi 3.4.2 hecd8cb5_6
libgdal 3.7.0 hc13fe4b_4 conda-forge libgfortran 5.0.0 11_3_0_h97931a8_31 conda-forge libgfortran5 12.2.0 he409387_31 conda-forge libglib 2.76.3 hc62aa5d_0 conda-forge libiconv 1.17 hac89ed1_0 conda-forge libjpeg-turbo 2.1.5.1 hb7f2c08_0 conda-forge libkml 1.3.0 haeb80ef_1015 conda-forge liblapack 3.9.0 17_osx64_openblas conda-forge libnetcdf 4.9.2 nompi_hfeda9e8_106 conda-forge libnghttp2 1.52.0 he2ab024_0 conda-forge libopenblas 0.3.23 openmp_h429af6e_0 conda-forge libpng 1.6.39 h6c40b1e_0
libpq 15.3 h9dc22bb_1 conda-forge librttopo 1.1.0 h23f359d_14 conda-forge libsodium 1.0.18 h1de35cc_0
libspatialite 5.0.1 h8e1b34b_28 conda-forge libsqlite 3.42.0 h58db7d2_0 conda-forge libssh2 1.11.0 hd019ec5_0 conda-forge libtiff 4.5.1 hf955e92_0 conda-forge libwebp-base 1.3.1 h0dc2134_0 conda-forge libxcb 1.15 hb7f2c08_0 conda-forge libxml2 2.11.4 hd95e348_0 conda-forge libxslt 1.1.37 h20bfa82_1 conda-forge libzip 1.9.2 h6db710c_1 conda-forge libzlib 1.2.13 h8a1eda9_5 conda-forge llvm-openmp 16.0.6 hff08bdf_0 conda-forge lxml 4.9.2 py310h479f746_1 conda-forge lz4-c 1.9.4 hf0c8a7f_0 conda-forge lzo 2.10 haf1e3a3_1000 conda-forge markupsafe 2.1.1 py310hca72f7f_0
matplotlib 3.7.1 py310h2ec42d9_0 conda-forge matplotlib-base 3.7.1 py310he725631_0 conda-forge matplotlib-inline 0.1.6 py310hecd8cb5_0
mistune 0.8.4 py310hca72f7f_1000
munkres 1.1.4 pyh9f0ad1d_0 conda-forge nbclassic 0.5.5 py310hecd8cb5_0
nbclient 0.5.13 py310hecd8cb5_0
nbconvert 6.5.4 py310hecd8cb5_0
nbformat 5.7.0 py310hecd8cb5_0
ncurses 6.4 hcec6c5f_0
nest-asyncio 1.5.6 py310hecd8cb5_0
netcdf4 1.6.4 nompi_py310h845552d_101 conda-forge notebook 6.5.4 py310hecd8cb5_0
notebook-shim 0.2.2 py310hecd8cb5_0
nspr 4.35 hea0b92c_0 conda-forge nss 3.89 h78b00b3_0 conda-forge numpy 1.25.0 py310h7451ae0_0 conda-forge openjpeg 2.5.0 h13ac156_2 conda-forge openssl 3.1.1 h8a1eda9_1 conda-forge packaging 23.0 py310hecd8cb5_0
pandas 2.0.3 py310h5e4fcda_0 conda-forge pandocfilters 1.5.0 pyhd3eb1b0_0
parso 0.8.3 pyhd3eb1b0_0
pcre2 10.40 h1c4e4bc_0 conda-forge pexpect 4.8.0 pyhd3eb1b0_3
pickleshare 0.7.5 pyhd3eb1b0_1003
pillow 10.0.0 py310hd63a8c7_0 conda-forge pip 23.0.1 py310hecd8cb5_0
pixman 0.40.0 hbcb3906_0 conda-forge platformdirs 2.5.2 py310hecd8cb5_0
pluggy 1.0.0 py310hecd8cb5_1
pooch 1.7.0 pyha770c72_3 conda-forge poppler 23.05.0 he041c3a_1 conda-forge poppler-data 0.4.12 hd8ed1ab_0 conda-forge postgresql 15.3 h325e403_1 conda-forge proj 9.2.1 hc8d59c9_0 conda-forge prometheus_client 0.14.1 py310hecd8cb5_0
prompt-toolkit 3.0.36 py310hecd8cb5_0
psutil 5.9.0 py310hca72f7f_0
pthread-stubs 0.4 hc929b4f_1001 conda-forge ptyprocess 0.7.0 pyhd3eb1b0_2
pure_eval 0.2.2 pyhd3eb1b0_0
pycosat 0.6.4 py310hca72f7f_0
pycparser 2.21 pyhd3eb1b0_0
pygments 2.15.1 py310hecd8cb5_1
pyopenssl 23.0.0 py310hecd8cb5_0
pyparsing 3.1.0 pyhd8ed1ab_0 conda-forge pyproj 3.6.0 py310h198f139_1 conda-forge pyrsistent 0.18.0 py310hca72f7f_0
pysocks 1.7.1 py310hecd8cb5_0
python 3.10.12 had23ca6_0_cpython conda-forge python-dateutil 2.8.2 pyhd3eb1b0_0
python-fastjsonschema 2.16.2 py310hecd8cb5_0
python-json-logger 2.0.7 py310hecd8cb5_0
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge python.app 3 py310hca72f7f_0
python_abi 3.10 2_cp310 conda-forge pytz 2022.7 py310hecd8cb5_0
pyyaml 6.0 py310h6c40b1e_1
pyzmq 25.1.0 py310hcec6c5f_0
rasterio 1.3.8 py310hd17acd7_0 conda-forge readline 8.2 hca72f7f_0
requests 2.28.1 py310hecd8cb5_1
rfc3339-validator 0.1.4 py310hecd8cb5_0
rfc3986-validator 0.1.1 py310hecd8cb5_0
rioxarray 0.14.1 pyhd8ed1ab_0 conda-forge ruamel.yaml 0.17.21 py310hca72f7f_0
ruamel.yaml.clib 0.2.6 py310hca72f7f_1
scipy 1.11.1 py310h3900cf1_0 conda-forge send2trash 1.8.0 pyhd3eb1b0_1
setuptools 65.6.3 py310hecd8cb5_0
six 1.16.0 pyhd3eb1b0_1
snappy 1.1.10 h225ccf5_0 conda-forge sniffio 1.2.0 py310hecd8cb5_1
snuggs 1.4.7 py_0 conda-forge soupsieve 2.4 py310hecd8cb5_0
sqlite 3.41.1 h6c40b1e_0
stack_data 0.2.0 pyhd3eb1b0_0
terminado 0.17.1 py310hecd8cb5_0
tiledb 2.13.2 h8b9cbf0_0 conda-forge tinycss2 1.2.1 py310hecd8cb5_0
tk 8.6.12 h5d9f67b_0
tomli 2.0.1 py310hecd8cb5_0
toolz 0.12.0 py310hecd8cb5_0
tornado 6.2 py310hca72f7f_0
tqdm 4.65.0 py310h20db666_0
traitlets 5.7.1 py310hecd8cb5_0
typing-extensions 4.6.3 py310hecd8cb5_0
typing_extensions 4.6.3 py310hecd8cb5_0
tzcode 2023c hb7f2c08_0 conda-forge tzdata 2023c h04d1e81_0
unicodedata2 15.0.0 py310h90acd4f_0 conda-forge urllib3 1.26.15 py310hecd8cb5_0
wcwidth 0.2.5 pyhd3eb1b0_0
webencodings 0.5.1 py310hecd8cb5_1
websocket-client 0.58.0 py310hecd8cb5_4
wheel 0.38.4 py310hecd8cb5_0
xarray 2023.6.0 pyhd8ed1ab_0 conda-forge xerces-c 3.2.4 h90c7484_2 conda-forge xorg-libxau 1.0.11 h0dc2134_0 conda-forge xorg-libxdmcp 1.1.3 h35c211d_0 conda-forge xz 5.2.10 h6c40b1e_1
y-py 0.5.9 py310h7242b5c_0
yaml 0.2.5 haf1e3a3_0
ypy-websocket 0.8.2 py310hecd8cb5_0
zeromq 4.3.4 h23ab428_0
zlib 1.2.13 h8a1eda9_5 conda-forge zstandard 0.19.0 py310h6c40b1e_0
zstd 1.5.5 hc035e20_0

[chiaravol]

How can I access each regionalized mean to plot them separately as subplot?

[malmans2]

The template already shows how to loop over each region:

for lats, da in da_daily.groupby("latitudes"):
    ...

If you need to access a specific one:

da_daily.sel(latitudes="(90, 60)")

For the plot you are looking for, ideally you could use xarray's one liner:

da_daily.plot(col="latitudes", col_wrap=3)  # use hue="latitudes" instead of col/col_wrap for a single plot

Because you also want to add filtered values, you might want to do something like this:

fig, axes = plt.subplots(3, 2, sharex=True, sharey=True)
for ax, (lats, da) in zip(axes.flatten(), da_daily.groupby("latitudes")):
    ax.plot(da["time"], da.values.squeeze(), label="daily")
    da_rolled = da.rolling(time=window, min_periods=1).mean()
    ax.plot(
        da_rolled["time"], da_rolled.values.squeeze(), label=f"{window}-day running"
    )
    ax.set_title(lats)
    ax.legend()

[chiaravol]

Thanks for your help Mattia!

Could you please run the code over the entire period? Here the notebook: ocean_color (1).ipynb.zip

[malmans2]

I've revised the template and cached the whole period available. Please use the latest template from now on. I've also added a couple of functions to plot the timeseries. Usage is: plot_daily(da, window=180, ...), where you can pass any plotting arguments.

Here is the template: https://github.com/bopen/c3s-eqc-toolbox-template/blob/main/notebooks/wp5/ocean_color.ipynb Here is the notebook executed: https://gist.github.com/malmans2/661f722c6c02473bbad74b11a586f5fd

Please let me know if everything works well.

[chiaravol]

It works well. Thank you