Enable Raven notebooks in nightly runs of Jenkins

tlvu commented 2 years ago

Description

Meta issue to link to other issues blocking this issue.

https://github.com/Ouranosinc/raven/issues/383

https://github.com/Ouranosinc/raven/issues/410

https://github.com/Ouranosinc/raven/issues/353

https://github.com/Ouranosinc/raven/pull/424

Also have to avoid the need to use --nbval-lax with Raven notebooks. This should be easier now that PR has been merged https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/73

Remember to test against test PAVICS server than production server. CRIM will spawn an fresh new PAVICS test server for each PR so support for test server is crucial to not break our automated testing pipeline. Any new test data needed by raven notebooks, need to be available on the test server.

tlvu commented 2 years ago

raven-nb-on-Jenkins.txt: Logs for Raven nb with --nbval-lax on Jenkins against production http://jenkins.ouranos.ca/job/PAVICS-e2e-workflow-tests/job/master/1477/consoleFull
Current Raven master: https://github.com/Ouranosinc/raven/commit/0d61774998d638d8f99b781d61d3a2fe7a0c2d28 on Jupyter docker image pavics/workflow-tests:220121.

8 failing notebooks:

10:49:40  =========================== short test summary info ============================
10:49:40  FAILED raven-master/docs/source/notebooks/03_Extract_geographical_watershed_properties.ipynb::Cell 6
10:49:40  FAILED raven-master/docs/source/notebooks/03_Extract_geographical_watershed_properties.ipynb::Cell 8
10:49:40  FAILED raven-master/docs/source/notebooks/03_Extract_geographical_watershed_properties.ipynb::Cell 9
10:49:40  FAILED raven-master/docs/source/notebooks/03_Extract_geographical_watershed_properties.ipynb::Cell 12
10:49:40  FAILED raven-master/docs/source/notebooks/05_Extracting_external_data.ipynb::Cell 2
10:49:40  FAILED raven-master/docs/source/notebooks/05_Extracting_external_data.ipynb::Cell 3
10:49:40  FAILED raven-master/docs/source/notebooks/06_Raven_calibration.ipynb::Cell 4
10:49:40  FAILED raven-master/docs/source/notebooks/06_Raven_calibration.ipynb::Cell 6
10:49:40  FAILED raven-master/docs/source/notebooks/06_Raven_calibration.ipynb::Cell 7
10:49:40  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 2
10:49:40  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 3
10:49:40  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 4
10:49:40  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 5
10:49:40  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 6
10:49:40  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 7
10:49:40  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 8
10:49:40  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 1
10:49:40  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 2
10:49:40  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 3
10:49:40  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 4
10:49:40  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 5
10:49:40  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 6
10:49:40  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 7
10:49:40  FAILED raven-master/docs/source/notebooks/12_Performing_hindcasting_experiments.ipynb::Cell 1
10:49:40  FAILED raven-master/docs/source/notebooks/12_Performing_hindcasting_experiments.ipynb::Cell 3
10:49:40  FAILED raven-master/docs/source/notebooks/12_Performing_hindcasting_experiments.ipynb::Cell 4
10:49:40  FAILED raven-master/docs/source/notebooks/12_Performing_hindcasting_experiments.ipynb::Cell 5
10:49:40  FAILED raven-master/docs/source/notebooks/Assess_probabilistic_flood_risk.ipynb::Cell 6
10:49:40  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 7
10:49:40  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 10
10:49:40  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 11
10:49:40  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 14
10:49:40  ================= 32 failed, 217 passed in 1243.60s (0:20:43) ==================

tlvu commented 2 years ago

Back in May 17, 2021, there was only 2 broken Raven notebooks on Jenkins, see comment https://github.com/bird-house/birdhouse-deploy/pull/170#issuecomment-842511189

All passed except 2 notebooks:

Assess_probabilistic_flood_risk.ipynb: I think due to missing thredds/fileServer/birdhouse/testdata/raven/raven-gr4j-cemaneige/Salmon-River-Near-Prince-George_meteo_daily.nc testdata. @cjauvin maybe switch this one to the dymamic way to get Raven testdata, like the rest of the notebooks?
Full_process_example_1.ipynb: known error, existing issue https://github.com/Ouranosinc/raven/issues/383

tlvu commented 2 years ago

Raven nb with --nbval-lax on Jenkins against production server after PR https://github.com/Ouranosinc/raven/pull/422 merged: http://jenkins.ouranos.ca/job/PAVICS-e2e-workflow-tests/job/master/1482/consoleFull Full log: build-1482-consoleText.txt

6 failing notebooks to go:

14:53:57  =========================== short test summary info ============================
14:53:57  FAILED raven-master/docs/source/notebooks/05_Extracting_external_data.ipynb::Cell 2
14:53:57  FAILED raven-master/docs/source/notebooks/05_Extracting_external_data.ipynb::Cell 3
CPLE_OpenFailedError: input.geojson: No such file or directory

14:53:57  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 2
14:53:57  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 3
14:53:57  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 4
14:53:57  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 6
14:53:57  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 7
14:53:57  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 8
14:53:57  FAILED raven-master/docs/source/notebooks/07_Bias_correction_of_CMIP6_data.ipynb::Cell 9
DriverError: input.geojson: No such file or directory

14:53:57  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 1
14:53:57  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 2
14:53:57  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 3
14:53:57  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 5
14:53:57  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 6
14:53:57  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 7
14:53:57  FAILED raven-master/docs/source/notebooks/09_Hydrological_impacts_of_climate_change.ipynb::Cell 8
CPLE_OpenFailedError: input.geojson: No such file or directory

14:53:57  FAILED raven-master/docs/source/notebooks/12_Performing_hindcasting_experiments.ipynb::Cell 1
14:53:57  FAILED raven-master/docs/source/notebooks/12_Performing_hindcasting_experiments.ipynb::Cell 3
14:53:57  FAILED raven-master/docs/source/notebooks/12_Performing_hindcasting_experiments.ipynb::Cell 4
14:53:57  FAILED raven-master/docs/source/notebooks/12_Performing_hindcasting_experiments.ipynb::Cell 5
DriverError: input.geojson: No such file or directory

14:53:57  FAILED raven-master/docs/source/notebooks/Assess_probabilistic_flood_risk.ipynb::Cell 6
AttributeError: 'Dataset' object has no attribute 'exceedance_probability'

14:53:57  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 7
14:53:57  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 10
14:53:57  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 11
14:53:57  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 14
AttributeError: 'bytes' object has no attribute 'where'

14:53:57  ================= 25 failed, 225 passed in 1459.11s (0:24:19) ==================

tlvu commented 2 years ago

@huard @richardarsenault To eventually run automatically under Jenkins, those S3 and Google drive credentials should be given to Jenkins.

There are 2 errors that seem to be code related:

14:53:57  FAILED raven-master/docs/source/notebooks/Assess_probabilistic_flood_risk.ipynb::Cell 6
AttributeError: 'Dataset' object has no attribute 'exceedance_probability'

14:53:57  FAILED raven-master/docs/source/notebooks/Extract_geographical_watershed_properties.ipynb::Cell 7
AttributeError: 'bytes' object has no attribute 'where'

See all details in previous comment https://github.com/Ouranosinc/raven/issues/423#issuecomment-1042338252.

Ouranosinc / raven

Enable Raven notebooks in nightly runs of Jenkins #423

Description