search

bird-house / birdhouse-deploy

Scripts and configurations to deploy the various birds and servers required for a full-fledged production platform
https://birdhouse-deploy.readthedocs.io/en/latest/
Apache License 2.0
4 stars 6 forks source link

jupyter: new incremental build for compatibility with upcoming Thredds v5 #420

Closed tlvu closed 7 months ago

tlvu commented 7 months ago

See https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/134 for more info.

Changes

Non-breaking changes

birdhouse_daccs_configs_branch: master birdhouse_skip_ci: false

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2418/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-jupyterenv
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-46.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1496/

NOTEBOOK TEST RESULTS 
    
[2024-01-17T19:46:00.267Z] ============================= test session starts ==============================
[2024-01-17T19:46:00.267Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-17T19:46:00.267Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-17T19:46:00.267Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-17T19:46:00.267Z] collected 264 items
[2024-01-17T19:46:00.267Z] 
[2024-01-17T19:46:12.128Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-17T19:46:52.385Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-17T19:46:57.057Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-17T19:47:06.022Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-17T19:47:16.095Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-17T19:47:23.955Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-01-17T19:54:43.888Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-17T19:54:43.888Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-17T19:54:48.895Z] ...............                                                          [ 33%]
[2024-01-17T19:54:58.303Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-17T19:55:05.686Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-17T19:55:21.685Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-17T19:55:31.972Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-17T19:55:36.674Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-17T20:00:35.252Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-17T20:01:50.935Z] .............                                                            [ 54%]
[2024-01-17T20:01:54.529Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-01-17T20:01:57.121Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-17T20:02:13.878Z] .................                                                        [ 65%]
[2024-01-17T20:02:21.655Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-17T20:02:23.037Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-17T20:02:50.651Z] ........F                                                                [ 71%]
[2024-01-17T20:03:01.565Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-17T20:03:12.040Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-17T20:03:13.953Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-17T20:03:17.030Z] ......                                                                   [ 81%]
[2024-01-17T20:03:23.629Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-17T20:03:39.625Z] .............                                                            [ 86%]
[2024-01-17T20:03:49.629Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-17T20:04:26.271Z] ....s.                                                                   [ 89%]
[2024-01-17T20:04:34.426Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-17T20:04:49.198Z] ...                                                                      [ 90%]
[2024-01-17T20:05:04.148Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-17T20:05:27.505Z] ......                                                                   [ 93%]
[2024-01-17T20:05:30.075Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-17T20:08:09.724Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-17T20:08:09.724Z] 
[2024-01-17T20:08:09.724Z] =================================== FAILURES ===================================
tlvu commented 7 months ago

I don't know enough about the workflow-tests image to have any useful insights

@mishaschwartz FYI

That workflow-tests image is actually our Jupyter runtime environment in production: https://pavics.ouranos.ca/jupyter ! Yeah I know bad name.

It was initially created as a Jenkins runtime test environment, hence the name, then I realized can run it in production as well so I did that to ensure we are deploying exactly what we tested. This way I am 100% sure all our tutorial notebooks work in production.

By the way, for UofT, which Jupyter env are you deploying? Or you are not enabling that component yet?

mishaschwartz commented 7 months ago

By the way, for UofT, which Jupyter env are you deploying? Or you are not enabling that component yet?

We have the defaults from env.local.example available but we're working on a jupyter image for general use that we're hoping can be included in other Marble nodes as an option as well.

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2429/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-jupyterenv
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-69.rdext.crim.ca

Infrastructure deployment failed. Instance has not been destroyed. @matprov

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2430/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-jupyterenv
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-69.rdext.crim.ca

Infrastructure deployment failed. Instance has not been destroyed. @matprov