search

bird-house / birdhouse-deploy

Scripts and configurations to deploy the various birds and servers required for a full-fledged production platform
https://birdhouse-deploy.readthedocs.io/en/latest/
Apache License 2.0
4 stars 7 forks source link

jupyter: new incremental build to include `SAlib` for sensitivity analysis and `fstd2nc` to convert RPN files (from Environment Canada) to netCDF files #398

Closed tlvu closed 11 months ago

tlvu commented 1 year ago

Also make /notebook_dir/ read-only to avoid users putting their files there and losing them since only /notebook_dir/writable-workspace is persisted on disk.

See https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/128 for more details about SAlib and https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/132 for more details about fstd2nc.

birdhouse_daccs_configs_branch: master birdhouse_skip_ci: false

crim-jenkins-bot commented 1 year ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2215/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : new-jupyter-env
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1398/

NOTEBOOK TEST RESULTS 
    
[2023-11-08T02:54:24.452Z] ============================= test session starts ==============================
[2023-11-08T02:54:24.452Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-11-08T02:54:24.452Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2023-11-08T02:54:24.452Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-11-08T02:54:24.452Z] collected 255 items
[2023-11-08T02:54:24.452Z] 
[2023-11-08T02:54:35.963Z] notebooks-auth/geoserver.ipynb ..................                        [  7%]
[2023-11-08T02:54:43.769Z] notebooks-auth/test_thredds.ipynb ...........                            [ 11%]
[2023-11-08T02:54:52.759Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 14%]
[2023-11-08T02:55:02.818Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 16%]
[2023-11-08T02:55:13.555Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 19%]
[2023-11-08T03:02:41.647Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 24%]
[2023-11-08T03:02:41.647Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 24%]
[2023-11-08T03:02:44.333Z] ...............                                                          [ 30%]
[2023-11-08T03:02:53.746Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb F.F..    [ 32%]
[2023-11-08T03:03:02.480Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 34%]
[2023-11-08T03:03:19.643Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 37%]
[2023-11-08T03:03:21.039Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 37%]
[2023-11-08T03:03:28.823Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 40%]
[2023-11-08T03:03:33.067Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 42%]
[2023-11-08T03:07:14.092Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 48%]
[2023-11-08T03:08:25.021Z] .............                                                            [ 53%]
[2023-11-08T03:08:26.411Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 54%]
[2023-11-08T03:08:29.155Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 58%]
[2023-11-08T03:08:45.095Z] .................                                                        [ 64%]
[2023-11-08T03:08:51.730Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 67%]
[2023-11-08T03:08:53.118Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 67%]
[2023-11-08T03:09:11.241Z] ........F                                                                [ 70%]
[2023-11-08T03:09:20.322Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 75%]
[2023-11-08T03:09:29.660Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 77%]
[2023-11-08T03:09:31.055Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2023-11-08T03:09:34.124Z] ......                                                                   [ 80%]
[2023-11-08T03:09:42.262Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 80%]
[2023-11-08T03:09:56.113Z] .............                                                            [ 85%]
[2023-11-08T03:10:04.573Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2023-11-08T03:10:43.879Z] ....s.                                                                   [ 88%]
[2023-11-08T03:10:52.025Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2023-11-08T03:11:04.552Z] ...                                                                      [ 90%]
[2023-11-08T03:11:19.481Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2023-11-08T03:11:39.517Z] ......                                                                   [ 92%]
[2023-11-08T03:11:41.992Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-11-08T03:14:21.614Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2023-11-08T03:14:21.614Z] 
[2023-11-08T03:14:21.615Z] =================================== FAILURES ===================================
crim-jenkins-bot commented 11 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2343/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : new-jupyter-env
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1465/

NOTEBOOK TEST RESULTS 
    
[2023-12-06T01:19:59.731Z] ============================= test session starts ==============================
[2023-12-06T01:19:59.731Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-12-06T01:19:59.731Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2023-12-06T01:19:59.731Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-12-06T01:19:59.731Z] collected 265 items
[2023-12-06T01:19:59.731Z] 
[2023-12-06T01:20:10.343Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2023-12-06T01:20:45.407Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2023-12-06T01:20:49.215Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2023-12-06T01:20:57.138Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2023-12-06T01:21:07.223Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2023-12-06T01:21:17.298Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2023-12-06T01:28:40.869Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2023-12-06T01:28:40.869Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2023-12-06T01:28:46.890Z] ...............                                                          [ 33%]
[2023-12-06T01:28:56.300Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2023-12-06T01:29:03.336Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2023-12-06T01:29:22.568Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2023-12-06T01:29:24.492Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 40%]
[2023-12-06T01:29:35.442Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2023-12-06T01:29:40.577Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2023-12-06T01:32:55.274Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2023-12-06T01:34:39.925Z] .............                                                            [ 55%]
[2023-12-06T01:34:42.383Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2023-12-06T01:34:44.697Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2023-12-06T01:35:04.534Z] .................                                                        [ 66%]
[2023-12-06T01:35:13.135Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2023-12-06T01:35:14.517Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2023-12-06T01:35:16.137Z] .FFFFFFFF                                                                [ 72%]
[2023-12-06T01:35:27.241Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2023-12-06T01:35:36.483Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2023-12-06T01:35:37.874Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2023-12-06T01:35:41.702Z] ......                                                                   [ 81%]
[2023-12-06T01:35:51.695Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2023-12-06T01:36:08.405Z] .............                                                            [ 86%]
[2023-12-06T01:36:18.404Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2023-12-06T01:36:55.986Z] ....s.                                                                   [ 89%]
[2023-12-06T01:37:04.108Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2023-12-06T01:37:19.637Z] ...                                                                      [ 90%]
[2023-12-06T01:37:34.536Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2023-12-06T01:37:59.076Z] ......                                                                   [ 93%]
[2023-12-06T01:38:00.539Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-12-06T01:40:34.694Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2023-12-06T01:40:34.694Z] 
[2023-12-06T01:40:34.694Z] =================================== FAILURES ===================================
crim-jenkins-bot commented 11 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2344/
Result : success

BIRDHOUSE_DEPLOY_BRANCH : new-jupyter-env
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-46.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1466/

NOTEBOOK TEST RESULTS 
    
[2023-12-06T01:27:38.003Z] ============================= test session starts ==============================
[2023-12-06T01:27:38.003Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-12-06T01:27:38.003Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master@2
[2023-12-06T01:27:38.003Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-12-06T01:27:38.003Z] collected 265 items
[2023-12-06T01:27:38.003Z] 
[2023-12-06T01:27:48.448Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2023-12-06T01:28:18.556Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2023-12-06T01:28:23.056Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2023-12-06T01:28:30.969Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2023-12-06T01:28:40.933Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2023-12-06T01:28:48.327Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2023-12-06T01:36:08.692Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2023-12-06T01:36:08.692Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2023-12-06T01:36:15.604Z] ...............                                                          [ 33%]
[2023-12-06T01:36:25.325Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2023-12-06T01:36:32.705Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2023-12-06T01:36:50.222Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2023-12-06T01:36:51.863Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 40%]
[2023-12-06T01:36:56.774Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2023-12-06T01:37:01.733Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2023-12-06T01:39:44.117Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2023-12-06T01:41:00.428Z] .............                                                            [ 55%]
[2023-12-06T01:41:04.319Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2023-12-06T01:41:06.645Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2023-12-06T01:41:23.509Z] .................                                                        [ 66%]
[2023-12-06T01:41:31.585Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2023-12-06T01:41:32.976Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2023-12-06T01:41:51.584Z] .........                                                                [ 72%]
[2023-12-06T01:42:00.831Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2023-12-06T01:42:10.582Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2023-12-06T01:42:12.496Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2023-12-06T01:42:15.300Z] ......                                                                   [ 81%]
[2023-12-06T01:42:23.437Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2023-12-06T01:42:37.377Z] .............                                                            [ 86%]
[2023-12-06T01:42:47.386Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2023-12-06T01:43:24.241Z] ....s.                                                                   [ 89%]
[2023-12-06T01:43:32.372Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2023-12-06T01:43:45.170Z] ...                                                                      [ 90%]
[2023-12-06T01:44:00.095Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2023-12-06T01:44:23.454Z] ......                                                                   [ 93%]
[2023-12-06T01:44:24.802Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-12-06T01:46:58.952Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2023-12-06T01:46:58.952Z] 
[2023-12-06T01:46:58.952Z] ================= 264 passed, 1 skipped in 1154.89s (0:19:14) ==================
crim-jenkins-bot commented 11 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2351/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : new-jupyter-env
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-46.rdext.crim.ca

Infrastructure deployment failed. Instance has not been destroyed. @matprov