search

Ouranosinc / PAVICS-e2e-workflow-tests

Test user-level workflow.
Apache License 2.0
0 stars 2 forks source link

Jupyter docker: new build for xclim, ravenpy, globus personal connect and latest of everything else #115

Closed tlvu closed 1 year ago

tlvu commented 1 year ago

Changes

< - cf_xarray=0.7.5=pyhd8ed1ab_0

  • cf_xarray=0.8.0=pyhd8ed1ab_0

< - clisops=0.9.3=pyh1a96a4e_0

  • clisops=0.9.6=pyh1a96a4e_0

< - dask=2022.11.1=pyhd8ed1ab_0

  • dask=2023.5.1=pyhd8ed1ab_0

< - esmf=8.2.0=mpi_mpich_h5a1934d_102 < - esmpy=8.2.0=mpi_mpich_py38h9147699_101

  • esmf=8.4.0=mpi_mpich_hc592774_104
  • esmpy=8.4.0=mpi_mpich_py39h3088dd8_102

< - fiona=1.8.22=py38h85a5e0f_5

  • fiona=1.9.1=py39hbc5ff6d_0

< - flox=0.6.4=pyhd8ed1ab_0

  • flox=0.7.2=pyhd8ed1ab_0

< - gdal=3.6.0=py38h58634bd_13

  • gdal=3.6.2=py39hc6cd174_6

< - owslib=0.27.2=pyhd8ed1ab_1

  • owslib=0.28.1=pyhd8ed1ab_0

new

  • pyogrio=0.5.1=py39hbc5ff6d_0

< - python=3.8.16=he550d4f_1_cpython

  • python=3.9.16=h2782a2a_0_cpython

< - raven-hydro=3.0.4.322=ha46ed4c_2 < - ravenpy=0.9.0=pyha21a80b_0

  • raven-hydro=0.2.1=py39h8e2dbb5_1
  • ravenpy=0.12.0=py39hf3d152e_2

< - rioxarray=0.13.1=pyhd8ed1ab_0

  • rioxarray=0.14.1=pyhd8ed1ab_0

< - roocs-utils=0.6.3=pyh1a96a4e_0

  • roocs-utils=0.6.4=pyh1a96a4e_0

new

  • seaborn=0.12.2=hd8ed1ab_0

< - xarray=2022.10.0=pyhd8ed1ab_0

  • xarray=2023.1.0=pyhd8ed1ab_0

< - xclim=0.39.0=pyhd8ed1ab_0

  • xclim=0.43.0=py39hf3d152e_1

< - xesmf=0.6.3=pyhd8ed1ab_1

  • xesmf=0.7.1=pyhd8ed1ab_0

new

  • hsclient==0.3.3


## Test

- Deployed as "beta" image in production for bokeh visualization performance regression testing.
- Manual test notebook https://github.com/Ouranosinc/PAVICS-landing/blob/master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb for bokeh visualization performance and it looks fine.
- Jenkins build, all passed:
  [job-PAVICS-e2e-workflow-tests-new-docker-build-198-consoleText.txt](https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/files/11621190/job-PAVICS-e2e-workflow-tests-new-docker-build-198-consoleText.txt)
  [ravenpy-job-PAVICS-e2e-workflow-tests-new-docker-build-200-consoleText.txt](https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/files/11621191/ravenpy-job-PAVICS-e2e-workflow-tests-new-docker-build-200-consoleText.txt)

## Related Issue / Discussion

- Matching notebook fixes:
  - Pavics-sdi: https://github.com/Ouranosinc/pavics-sdi/pull/281
  - Finch: https://github.com/bird-house/finch/pull/275
  - PAVICS-landing: https://github.com/Ouranosinc/PAVICS-landing/pull/64
  - RavenPy: https://github.com/CSHS-CWRA/RavenPy/pull/287

- Deployment to PAVICS: https://github.com/bird-house/birdhouse-deploy/pull/332

- Jenkins-config changes for new RavenPy notebooks: https://github.com/Ouranosinc/jenkins-config/pull/15

- Other issues found while working on this one
  - https://github.com/Ouranosinc/PAVICS-landing/issues/66
  - https://github.com/Ouranosinc/PAVICS-landing/issues/65

- Previous release: https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/118

## Additional Information

Full diff conda env export:
[221130-update230403-230530-1-conda-env-export.diff.txt](https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/files/11621128/221130-update230403-230530-1-conda-env-export.diff.txt)

Full new conda env export:
[230530-1-conda-env-export.yml.txt](https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/files/11621130/230530-1-conda-env-export.yml.txt)

Docker build log
[230530-1-buildlogs.txt](https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/files/11621136/230530-1-buildlogs.txt)
Zeitsperre commented 1 year ago

@tlvu

You might want to grab the newest xclim while you're at it (xclim-0.41.0-pyhd8ed1ab_0)

tlvu commented 1 year ago

@tlvu

You might want to grab the newest xclim while you're at it (xclim-0.41.0-pyhd8ed1ab_0)

@Zeitsperre yup yup that is exactly what commit eab04b5062ac42bcde21cd316d4d95ab3576b810 is doing :D

tlvu commented 1 year ago

Since this PR pulls a new RavenPy that is incompatible with the existing RavenWPS, we need to sync the go-live of this new Jupyter env together with a new RavenWPS also. But the new RavenWPS docker is failing to start ! Under investigation. The go-live of this PR will have to be delayed a bit.