Jupyter env rebuild

[tlvu]

Overview

Periodic rebuild to update the Jupyter env.

Changes

Relevant changes:

<   - xclim=0.28.1=pyhd8ed1ab_0
>   - xclim=0.31.0=pyhd8ed1ab_0

<   - ravenpy=0.7.4=pyh7f9bfb9_0
>   - ravenpy=0.7.5=pyhff6ddc9_0

<   - xarray=0.19.0=pyhd8ed1ab_1
>   - xarray=0.20.1=pyhd8ed1ab_0

<   - rasterio=1.2.1=py37ha549118_0
>   - rasterio=1.2.6=py37hc20819c_2  

<   - bokeh=2.3.3=py37h89c1867_0
>   - bokeh=2.4.2=py37h89c1867_0

<   - cartopy=0.19.0.post1=py37h0c48da3_1
>   - cartopy=0.20.0=py37hbe109c4_0

<   - cffi=1.14.6=py37hc58025e_0
>   - cffi=1.15.0=py37h036bc23_0

<   - climpred=2.1.5.post1=pyhd8ed1ab_0
>   - climpred=2.1.6=pyhd8ed1ab_1

<   - clisops=0.6.5=pyh6c4a22f_0
>   - clisops=0.7.0=pyh6c4a22f_0

<   - dask=2021.9.0=pyhd8ed1ab_0
>   - dask=2021.11.2=pyhd8ed1ab_0

<   - gdal=3.1.4=py37h2ec2946_8
>   - gdal=3.3.2=py37hd5a0ba4_2

<   - geopandas=0.9.0=pyhd8ed1ab_1
>   - geopandas=0.10.2=pyhd8ed1ab_0

<   - nc-time-axis=1.3.1=pyhd8ed1ab_2
>   - nc-time-axis=1.4.0=pyhd8ed1ab_0

<   - pandas=1.2.5=py37h219a48f_0
>   - pandas=1.3.4=py37he8f5f7f_

<   - poppler=0.89.0=h2de54a5_5
>   - poppler=21.09.0=ha39eefc_3

<   - rioxarray=0.7.0=pyhd8ed1ab_0
>   - rioxarray=0.8.0=pyhd8ed1ab_0

<   - roocs-utils=0.4.2=pyh6c4a22f_0
>   - roocs-utils=0.5.0=pyh6c4a22f_0

Full diff of conda env export: 210908-211123-conda-env-export.diff.txt

211123-211123-update211216-conda-env-export.diff.txt

210908-211123-update211216-conda-env-export.diff.txt

Full new conda env export: 211123-conda-env-export.yml.txt

211123-update211216-conda-env-export.yml.txt

Related Issue / Discussion

• Corresponding notebook update https://github.com/Ouranosinc/pavics-sdi/pull/241
• Corresponding deployment to PAVICS https://github.com/bird-house/birdhouse-deploy/pull/229

Additional Information

Jenkins build with only known intermittent failure http://jenkins.ouranos.ca/job/PAVICS-e2e-workflow-tests/job/jupyter-env-rebuild/8/console

[tlvu]

@Zeitsperre note RavenPy is downgraded !

[tlvu]

@Zeitsperre note RavenPy is downgraded !

Suspect due to rasterio pinning in ravenpy 0.7.5 https://github.com/conda-forge/ravenpy-feedstock/pull/14/commits/4140d6e3c1fade20f82c02ee55b5cc06320d8591

[tlvu]

@Zeitsperre note RavenPy is downgraded !

Suspect due to rasterio pinning in ravenpy 0.7.5 conda-forge/ravenpy-feedstock@4140d6e

Confirmed due to rasterio pin in ravenpy. Fixed as well. We have ravenpy 0.7.5 now.

[Zeitsperre]

We need to adjust some of these libraries (fiona and rasterio at the very least) in a new ravenpy. I can't remember what was blocking them, but I seem to recall that both had API changes. To be determined.

[tlvu]

We need to adjust some of these libraries (fiona and rasterio at the very least) in a new ravenpy. I can't remember what was blocking them, but I seem to recall that both had API changes. To be determined.

@Zeitsperre @tlogan2000 @huard

We gotta find a way for ravenpy and xclim conda package to take care of their own pinning so I do not have to manually duplicate all the pinning in here (and have to remove those pinning later as well). Every rebuild since a year is a pinning land mine for me since I always discover pinning too late. Releasing a new Jupyter env has becoming much more labor intensive.

This rasterio is a weird case. The latest ravenpy 0.7.5 conda has rasterio pinned properly but I still had to perform the same manual pin to pickup the latest ravenpy.

However here is a pinning out of sync for ravenpy between the source and the conda package at the moment of this writing:

conda package no shapely pin: https://github.com/conda-forge/ravenpy-feedstock/blob/88efd89f7650f4dddab846c022696d53262eed36/recipe/meta.yaml#L49

source has shapely pin: https://github.com/CSHS-CWRA/RavenPy/blob/f63e1e5b967c0d7c17e679c8f9d6d309a94096e6/environment.yml#L35

I am thinking to remove all packages declared in the Jupyter env if they are direct dependencies of ravenpy and xclim. Hopefully this way the pinning inside ravenpy and xclim will take effect. Do you think this would work? Any other suggestions?

For the out-of-sync of dependencies pinning between the source and the conda package, is there a way for the the conda package to share a common dependencies file in the source repo? This way we avoid duplicating pinning between the 2 as well.