bird-house / birdhouse-deploy

Scripts and configurations to deploy the various birds and servers required for a full-fledged production platform

https://birdhouse-deploy.readthedocs.io/en/latest/

Apache License 2.0

4 stars 6 forks source link

decrease optional var logging level and allow to pass extra opts to `pavics-compose.sh up -d` #426

Closed tlvu closed 7 months ago

tlvu commented 7 months ago

logging: decrease logging level for empty optional vars from WARN to DEBUG

To avoid drowning real WARN messages. Many optional vars can be valid if empty.
config: add sample config to configure docker-compose to remove orphans

To remove orphans containers when components are disabled. Also link to full documentations if other env var can be used.
compose script: allow to pass extra options to up operation

The previous docker-compose built-in env var was not working so had to add this homegrown solution.

When disabling components, their existing containers will not be removed unless option --remove-orphans is given together with ./pavics-compose.sh up -d.

This change allow any additional options, not just --remove-orphans.
compose script: exit early when any errors occurred during invocation

Before, all the post-docker-compose-up would still execute after docker-compose has an error.

tlvu commented 7 months ago

FYI I added another very small change. PR title and description updated.

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2498/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1532/

NOTEBOOK TEST RESULTS

    
[2024-02-23T21:51:13.086Z] ============================= test session starts ==============================
[2024-02-23T21:51:13.086Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-23T21:51:13.086Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-23T21:51:13.086Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-23T21:51:13.086Z] collected 264 items
[2024-02-23T21:51:13.086Z] 
[2024-02-23T21:51:24.001Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-23T21:51:57.962Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-23T21:52:05.051Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-23T21:52:14.844Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-23T21:52:25.700Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-23T21:52:31.023Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 22%]
[2024-02-23T22:00:33.215Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-23T22:00:33.215Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-23T22:00:42.421Z] ...............                                                          [ 33%]
[2024-02-23T22:00:51.847Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-23T22:06:56.327Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb FFFFFF            [ 37%]
[2024-02-23T22:07:14.491Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-23T22:07:20.337Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-23T22:07:24.849Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-23T22:11:06.162Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-23T22:12:52.376Z] .............                                                            [ 54%]
[2024-02-23T22:12:56.460Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-23T22:12:59.461Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-23T22:13:26.622Z] .................                                                        [ 65%]
[2024-02-23T22:13:35.305Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-23T22:13:37.224Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-23T22:13:55.606Z] .........                                                                [ 71%]
[2024-02-23T22:14:04.962Z] finch-master/docs/source/notebooks/dap_subset.ipynb F........F.          [ 76%]
[2024-02-23T22:14:15.497Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-23T22:14:18.029Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-23T22:14:23.773Z] ......                                                                   [ 81%]
[2024-02-23T22:15:20.028Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-23T22:16:09.947Z] .............                                                            [ 86%]
[2024-02-23T22:16:19.975Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-23T22:17:00.168Z] ....s.                                                                   [ 89%]
[2024-02-23T22:17:08.295Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-23T22:17:23.812Z] ...                                                                      [ 90%]
[2024-02-23T22:17:38.734Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-23T22:18:00.454Z] ......                                                                   [ 93%]
[2024-02-23T22:18:02.932Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-23T22:20:37.042Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-23T22:20:37.042Z] 
[2024-02-23T22:20:37.042Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2499/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-133.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1533/

NOTEBOOK TEST RESULTS

    
[2024-02-23T22:08:38.640Z] ============================= test session starts ==============================
[2024-02-23T22:08:38.640Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-23T22:08:38.640Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master@2
[2024-02-23T22:08:38.640Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-23T22:08:38.640Z] collected 264 items
[2024-02-23T22:08:38.640Z] 
[2024-02-23T22:08:48.002Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-23T22:09:12.959Z] notebooks-auth/test_cowbird_jupyter.ipynb ........F.                     [ 10%]
[2024-02-23T22:09:21.226Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-23T22:09:30.185Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-23T22:09:39.903Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-23T22:09:43.531Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ......FF       [ 22%]
[2024-02-23T22:17:31.294Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-23T22:17:33.216Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-23T22:17:40.946Z] ...............                                                          [ 33%]
[2024-02-23T22:17:50.347Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-23T22:23:55.161Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb FFFFFF            [ 37%]
[2024-02-23T22:24:13.057Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-23T22:24:18.229Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-23T22:24:22.487Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-23T22:27:01.850Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-23T22:28:19.419Z] .............                                                            [ 54%]
[2024-02-23T22:28:21.602Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-23T22:28:23.931Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-23T22:28:38.668Z] .................                                                        [ 65%]
[2024-02-23T22:28:46.563Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-23T22:28:47.942Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-23T22:28:59.555Z] .........                                                                [ 71%]
[2024-02-23T22:29:10.379Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-23T22:29:19.287Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-23T22:29:20.669Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-23T22:29:23.741Z] ......                                                                   [ 81%]
[2024-02-23T22:29:30.325Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-23T22:29:45.248Z] .............                                                            [ 86%]
[2024-02-23T22:29:55.271Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-23T22:30:39.540Z] ....s.                                                                   [ 89%]
[2024-02-23T22:30:47.673Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-23T22:31:02.448Z] ...                                                                      [ 90%]
[2024-02-23T22:31:14.692Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-23T22:31:36.650Z] ......                                                                   [ 93%]
[2024-02-23T22:31:38.896Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-23T22:34:12.997Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-23T22:34:12.997Z] 
[2024-02-23T22:34:12.997Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2501/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1535/

NOTEBOOK TEST RESULTS

    
[2024-02-24T04:59:20.659Z] ============================= test session starts ==============================
[2024-02-24T04:59:20.659Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-24T04:59:20.659Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-24T04:59:20.659Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-24T04:59:20.659Z] collected 264 items
[2024-02-24T04:59:20.659Z] 
[2024-02-24T04:59:32.196Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-24T05:00:05.891Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-24T05:00:10.665Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-24T05:00:20.543Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-24T05:00:29.659Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-24T05:00:40.737Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-02-24T05:07:53.647Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-24T05:07:56.189Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-24T05:08:04.330Z] ...............                                                          [ 33%]
[2024-02-24T05:08:14.058Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-24T05:14:18.614Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb FFFFFF            [ 37%]
[2024-02-24T05:14:36.387Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-24T05:14:42.049Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-24T05:14:46.563Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-24T05:18:07.205Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-24T05:19:24.536Z] .............                                                            [ 54%]
[2024-02-24T05:19:26.977Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-24T05:19:29.201Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-24T05:19:46.470Z] .................                                                        [ 65%]
[2024-02-24T05:19:53.670Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-24T05:19:55.583Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-24T05:20:13.633Z] .........                                                                [ 71%]
[2024-02-24T05:20:22.873Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-24T05:20:32.213Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-24T05:20:33.605Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-24T05:20:36.905Z] ......                                                                   [ 81%]
[2024-02-24T05:20:45.063Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-24T05:20:59.167Z] .............                                                            [ 86%]
[2024-02-24T05:21:09.169Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-24T05:21:43.129Z] ....s.                                                                   [ 89%]
[2024-02-24T05:21:51.287Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-24T05:22:07.097Z] ...                                                                      [ 90%]
[2024-02-24T05:22:22.016Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-24T05:22:45.413Z] ......                                                                   [ 93%]
[2024-02-24T05:22:47.451Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-24T05:25:22.157Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-24T05:25:22.157Z] 
[2024-02-24T05:25:22.157Z] =================================== FAILURES ===================================

tlvu commented 7 months ago

Given this point (#426 (comment)) I think that the COMPOSE_EXTRA_OPTS has the potential to cause a lot of confusion.

I think that a better strategy is to encourage users to export any of the environment variables from
* https://docs.docker.com/compose/environment-variables/envvars/

* https://docs.docker.com/engine/reference/commandline/cli/#environment-variables
in their env.local file instead.

For example, instead of setting:
export COMPOSE_UP_EXTRA_OPTS="--remove-orphans"
they would set:
export COMPOSE_REMOVE_ORPHANS=true
Not all CLI options/flags are available as environment variables but I think that enough are that we can cover at least the basic use-cases and this is a well documented feature of docker compose.

Also, the precedence is clearly documented here: https://docs.docker.com/compose/environment-variables/envvars-precedence/ so we can avoid some confusion.

True ! If docker-compose already have some built-in env var to solve my problem, why re-invent the wheel.

It does not cover all the options yet, but it's probably enough for now.

I'll just document this in env.local.example then.

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2505/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1539/

NOTEBOOK TEST RESULTS

    
[2024-02-27T20:01:02.280Z] ============================= test session starts ==============================
[2024-02-27T20:01:02.280Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-27T20:01:02.280Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-27T20:01:02.280Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-27T20:01:02.280Z] collected 264 items
[2024-02-27T20:01:02.280Z] 
[2024-02-27T20:01:13.756Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-27T20:01:48.548Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-27T20:01:52.795Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-27T20:02:02.018Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-27T20:02:12.123Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-27T20:02:24.901Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-02-27T20:10:00.327Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-27T20:10:00.328Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-27T20:11:55.827Z] ...............                                                          [ 33%]
[2024-02-27T20:12:05.100Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-27T20:12:12.202Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-02-27T20:12:28.748Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-27T20:12:33.840Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-27T20:12:38.603Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-27T20:15:51.658Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-27T20:17:09.942Z] .............                                                            [ 54%]
[2024-02-27T20:17:12.383Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-27T20:17:14.639Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-27T20:17:30.361Z] .................                                                        [ 65%]
[2024-02-27T20:17:38.435Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-27T20:17:39.816Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-27T20:18:53.621Z] ......FFF                                                                [ 71%]
[2024-02-27T20:19:00.198Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-27T20:19:09.195Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-27T20:19:10.579Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-27T20:19:13.657Z] ......                                                                   [ 81%]
[2024-02-27T20:19:21.785Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-27T20:19:37.444Z] .............                                                            [ 86%]
[2024-02-27T20:19:47.438Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-27T20:20:44.160Z] ....s.                                                                   [ 89%]
[2024-02-27T20:20:52.297Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-27T20:21:07.073Z] ...                                                                      [ 90%]
[2024-02-27T20:21:19.324Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-27T20:21:41.699Z] ......                                                                   [ 93%]
[2024-02-27T20:21:44.433Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-27T20:24:24.588Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-27T20:24:24.588Z] 
[2024-02-27T20:24:24.588Z] =================================== FAILURES ===================================

tlvu commented 7 months ago

@mishaschwartz @fmigneault

Saddly, I tested and COMPOSE_REMOVE_ORPHANS do not work. A bit of search found this https://github.com/docker/compose/issues/11374.

So how about having COMPOSE_UP_EXTRA_OPTS as a "hidden" option not documented in env.local.example? We have a few hidden options already so this is not a precedence.

mishaschwartz commented 7 months ago

So how about having COMPOSE_UP_EXTRA_OPTS as a "hidden" option

I don't think it needs to be hidden, but we need a better way of telling the user what the cause of an error is. Maybe something like:

log INFO "Executing docker-compose with options: ${COMPOSE_UP_EXTRA_OPTS} $*"
PROXY_SECURE_PORT=443 HOSTNAME=${PAVICS_FQDN} docker-compose ${COMPOSE_CONF_LIST} ${COMPOSE_UP_EXTRA_OPTS} $*
ERR=$?
[ ${ERR} -gt 0 ] && exit ${ERR}

fmigneault commented 7 months ago

@mishaschwartz's proposal seems fine to me. As long as there's a way to figure out from the logs why the command could have failed.

tlvu commented 7 months ago

@mishaschwartz's proposal seems fine to me. As long as there's a way to figure out from the logs why the command could have failed.

Great, so I will un-backout my previous commit :)

Luckily I did not rush to merge this "simple" change on Friday.

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2507/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1541/

NOTEBOOK TEST RESULTS

    
[2024-02-28T19:47:44.337Z] ============================= test session starts ==============================
[2024-02-28T19:47:44.337Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-28T19:47:44.337Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-28T19:47:44.337Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-28T19:47:44.337Z] collected 264 items
[2024-02-28T19:47:44.337Z] 
[2024-02-28T19:47:55.101Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-28T19:48:35.333Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-28T19:48:39.583Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-28T19:48:48.919Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-28T19:48:58.640Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-28T19:49:01.626Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 22%]
[2024-02-28T19:56:38.915Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-28T19:56:40.845Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-28T19:56:49.545Z] ...............                                                          [ 33%]
[2024-02-28T19:56:58.606Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-28T19:57:06.343Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-02-28T19:57:22.753Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-28T19:57:35.877Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-28T19:57:40.830Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-28T20:01:07.452Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-28T20:02:25.949Z] .............                                                            [ 54%]
[2024-02-28T20:02:28.660Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-28T20:02:30.817Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-28T20:02:54.476Z] .................                                                        [ 65%]
[2024-02-28T20:03:02.686Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-28T20:03:04.065Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-28T20:03:22.393Z] .........                                                                [ 71%]
[2024-02-28T20:03:33.303Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-28T20:03:42.591Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-28T20:03:43.970Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-28T20:03:47.059Z] ......                                                                   [ 81%]
[2024-02-28T20:03:55.203Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-28T20:04:11.859Z] .............                                                            [ 86%]
[2024-02-28T20:04:21.849Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-28T20:05:05.164Z] ....s.                                                                   [ 89%]
[2024-02-28T20:05:13.302Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-28T20:05:28.827Z] ...                                                                      [ 90%]
[2024-02-28T20:05:41.347Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-28T20:06:03.110Z] ......                                                                   [ 93%]
[2024-02-28T20:06:05.597Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-28T20:08:39.699Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-28T20:08:39.699Z] 
[2024-02-28T20:08:39.699Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2508/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-133.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1542/

NOTEBOOK TEST RESULTS

    
[2024-02-28T20:10:30.140Z] ============================= test session starts ==============================
[2024-02-28T20:10:30.140Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-28T20:10:30.140Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-28T20:10:30.140Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-28T20:10:30.140Z] collected 264 items
[2024-02-28T20:10:30.140Z] 
[2024-02-28T20:10:41.221Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-28T20:11:10.351Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-28T20:11:14.838Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-28T20:11:23.748Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-28T20:11:32.137Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-28T20:11:40.028Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 22%]
[2024-02-28T20:20:26.877Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-28T20:20:26.877Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-28T20:20:28.774Z] ...............                                                          [ 33%]
[2024-02-28T20:20:37.815Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-28T20:20:44.508Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-02-28T20:21:00.925Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-28T20:21:07.110Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-28T20:21:14.347Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-28T20:24:00.286Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-28T20:25:18.092Z] .............                                                            [ 54%]
[2024-02-28T20:25:20.521Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-28T20:25:22.843Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-28T20:25:39.797Z] .................                                                        [ 65%]
[2024-02-28T20:25:46.978Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-28T20:25:48.363Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-28T20:26:06.675Z] .........                                                                [ 71%]
[2024-02-28T20:26:18.416Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-28T20:26:26.935Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-28T20:26:28.320Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-28T20:26:31.383Z] ......                                                                   [ 81%]
[2024-02-28T20:26:39.516Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-28T20:26:55.898Z] .............                                                            [ 86%]
[2024-02-28T20:27:05.891Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-28T20:27:53.584Z] ....s.                                                                   [ 89%]
[2024-02-28T20:28:03.579Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-28T20:28:17.706Z] ...                                                                      [ 90%]
[2024-02-28T20:28:32.601Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-28T20:28:55.498Z] ......                                                                   [ 93%]
[2024-02-28T20:28:57.798Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-28T20:31:43.434Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-28T20:31:43.434Z] 
[2024-02-28T20:31:43.434Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2509/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1543/

NOTEBOOK TEST RESULTS

    
[2024-02-28T20:28:01.085Z] ============================= test session starts ==============================
[2024-02-28T20:28:01.085Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-28T20:28:01.085Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master@2
[2024-02-28T20:28:01.085Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-28T20:28:01.085Z] collected 264 items
[2024-02-28T20:28:01.085Z] 
[2024-02-28T20:28:12.103Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-28T20:28:48.011Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-28T20:28:53.269Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-28T20:29:01.672Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-28T20:29:11.384Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-28T20:29:18.066Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-02-28T20:36:50.859Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-28T20:36:51.435Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-28T20:36:59.839Z] ...............                                                          [ 33%]
[2024-02-28T20:37:09.116Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-28T20:37:16.307Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-02-28T20:37:33.065Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-28T20:37:38.169Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-28T20:37:42.953Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-28T20:40:40.281Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-28T20:41:47.098Z] .............                                                            [ 54%]
[2024-02-28T20:41:52.091Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-28T20:41:53.966Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-28T20:42:10.066Z] .................                                                        [ 65%]
[2024-02-28T20:42:18.413Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-28T20:42:19.798Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-28T20:42:43.748Z] .........                                                                [ 71%]
[2024-02-28T20:42:52.605Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-28T20:43:02.600Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-28T20:43:03.981Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-28T20:43:07.054Z] ......                                                                   [ 81%]
[2024-02-28T20:43:15.201Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-28T20:43:30.797Z] .............                                                            [ 86%]
[2024-02-28T20:43:40.816Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-28T20:44:28.554Z] ....s.                                                                   [ 89%]
[2024-02-28T20:44:36.698Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-28T20:44:52.251Z] ...                                                                      [ 90%]
[2024-02-28T20:45:04.492Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-28T20:45:27.510Z] ......                                                                   [ 93%]
[2024-02-28T20:45:29.989Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-28T20:48:09.615Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-28T20:48:09.615Z] 
[2024-02-28T20:48:09.616Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2521/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1546/

NOTEBOOK TEST RESULTS

    
[2024-03-06T22:19:12.907Z] ============================= test session starts ==============================
[2024-03-06T22:19:12.907Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-03-06T22:19:12.907Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-03-06T22:19:12.907Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-03-06T22:19:12.907Z] collected 264 items
[2024-03-06T22:19:12.907Z] 
[2024-03-06T22:19:26.249Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-03-06T22:20:00.954Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-03-06T22:20:07.776Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-03-06T22:20:18.125Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-03-06T22:20:30.082Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-03-06T22:20:35.999Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-03-06T22:28:35.736Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-03-06T22:28:35.736Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-03-06T22:28:40.348Z] ...............                                                          [ 33%]
[2024-03-06T22:28:50.254Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-03-06T22:29:35.097Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-03-06T22:29:52.651Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-03-06T22:30:19.922Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-03-06T22:30:24.876Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-03-06T22:30:33.242Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb EEEEEEEEEEEEEEE [ 50%]
[2024-03-06T22:30:43.718Z] EEEEEEEEEEEEE                                                            [ 54%]
[2024-03-06T22:30:48.930Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-03-06T22:30:52.306Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-03-06T22:31:20.472Z] .................                                                        [ 65%]
[2024-03-06T22:31:30.798Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-03-06T22:31:32.718Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-03-06T22:31:38.585Z] .FFFFFFFF                                                                [ 71%]
[2024-03-06T22:31:51.507Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-03-06T22:32:01.972Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-03-06T22:32:04.527Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-03-06T22:32:10.181Z] ......                                                                   [ 81%]
[2024-03-06T22:33:06.463Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-03-06T22:34:03.053Z] .............                                                            [ 86%]
[2024-03-06T22:34:17.960Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-03-06T22:36:22.217Z] ....s.                                                                   [ 89%]
[2024-03-06T22:36:30.365Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-03-06T22:36:46.813Z] ...                                                                      [ 90%]
[2024-03-06T22:37:01.714Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-03-06T22:37:26.235Z] ......                                                                   [ 93%]
[2024-03-06T22:37:29.115Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-03-06T22:40:09.300Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-03-06T22:40:09.300Z] 
[2024-03-06T22:40:09.300Z] ==================================== ERRORS ====================================

crim-jenkins-bot commented 7 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2522/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-optional-vars-log-level
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-133.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1547/

NOTEBOOK TEST RESULTS

    
[2024-03-06T22:28:14.961Z] ============================= test session starts ==============================
[2024-03-06T22:28:14.961Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-03-06T22:28:14.961Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master@2
[2024-03-06T22:28:14.961Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-03-06T22:28:14.961Z] collected 264 items
[2024-03-06T22:28:14.961Z] 
[2024-03-06T22:28:26.235Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-03-06T22:28:56.624Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-03-06T22:29:01.399Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-03-06T22:29:10.367Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-03-06T22:29:20.106Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-03-06T22:29:26.569Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-03-06T22:37:22.044Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-03-06T22:37:24.600Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-03-06T22:37:34.028Z] ...............                                                          [ 33%]
[2024-03-06T22:37:44.210Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-03-06T22:38:36.062Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-03-06T22:38:49.814Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-03-06T22:38:55.098Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-03-06T22:39:00.233Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-03-06T22:42:32.479Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-03-06T22:43:48.781Z] .............                                                            [ 54%]
[2024-03-06T22:43:50.248Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-03-06T22:43:52.571Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-03-06T22:44:09.036Z] .................                                                        [ 65%]
[2024-03-06T22:44:16.164Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-03-06T22:44:17.548Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-03-06T22:44:35.321Z] .........                                                                [ 71%]
[2024-03-06T22:44:45.127Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-03-06T22:44:54.148Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-03-06T22:44:56.055Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-03-06T22:44:59.115Z] ......                                                                   [ 81%]
[2024-03-06T22:45:09.110Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-03-06T22:45:33.296Z] .............                                                            [ 86%]
[2024-03-06T22:45:43.305Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-03-06T22:46:25.402Z] ....s.                                                                   [ 89%]
[2024-03-06T22:46:33.529Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-03-06T22:46:46.044Z] ...                                                                      [ 90%]
[2024-03-06T22:47:01.083Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-03-06T22:47:21.927Z] ......                                                                   [ 93%]
[2024-03-06T22:47:24.406Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-03-06T22:50:04.285Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-03-06T22:50:04.285Z] 
[2024-03-06T22:50:04.285Z] =================================== FAILURES ===================================