bird-house / birdhouse-deploy

Scripts and configurations to deploy the various birds and servers required for a full-fledged production platform

https://birdhouse-deploy.readthedocs.io/en/latest/

Apache License 2.0

4 stars 7 forks source link

node details overrides + logging utils #415

Closed fmigneault closed 9 months ago

fmigneault commented 11 months ago

Overview

Add server details overrides and logging utilities.

Changes

Non-breaking changes

Compose script utilities:
- Add BIRDHOUSE_COLOR option and various logging/messaging definitions in birdhouse/scripts/logging.include.sh.
- Replace all explicit color "logging" related echo in scripts by utility variables MSG_DEBUG, MSG_INFO, MSG_WARN and MSG_ERROR as applicable per respective messages.
- Unify all birdhouse/scripts utilities to employ the same COMPOSE_DIR variable (auto-resolved or explicitly set) in order to include or source any relevant dependencies they might have within the birdhouse-deploy repository.
- Add info option (ie: pavics-compose.sh info) that will stop processing just before docker-compose call. This can be used to run a "dry-run" of the command and validate that was is loaded is as expected, by inspecting provided log messages.
Defaults:
- Add multiple SERVER_[...] variables with defaults using previously hard coded values referring to PAVICS. These variables use a special combination of DELAYED_EVAL and OPTIONAL_VARS definitions that can make use of a variable formatted as <ANY_NAME>='${__DEFAULT__<ANY_NAME>}' that will print a warning messages indicating that the default is employed, although STRONGLY recommended to be overridden. This allows a middle ground between backward-compatible env.local while flagging potentially misused configurations.
Canarie-API: updated references
- Use the new SERVER_[...] variables.
- Replace the LICENSE URL of the server node pointing at Ouranosinc/pavics-sdi instead of intended bird-house/birdhouse-deploy.

Breaking changes

Related Issue / Discussion

Partially related to #414

CI

birdhouse_daccs_configs_branch: master birdhouse_skip_ci: false

crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2369/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-90.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1486/

NOTEBOOK TEST RESULTS

    
[2024-01-09T19:20:03.221Z] ============================= test session starts ==============================
[2024-01-09T19:20:03.221Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-09T19:20:03.221Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-09T19:20:03.221Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-09T19:20:03.221Z] collected 265 items
[2024-01-09T19:20:03.221Z] 
[2024-01-09T19:20:30.385Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-09T19:21:26.276Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-09T19:22:14.876Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-09T19:22:25.855Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-09T19:22:35.939Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-09T19:22:45.463Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-01-09T19:34:56.555Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-09T19:34:59.865Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-09T19:35:35.621Z] ...............                                                          [ 33%]
[2024-01-09T19:35:46.948Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-09T19:35:54.302Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-09T19:36:11.434Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-09T19:36:13.112Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 40%]
[2024-01-09T19:36:18.453Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-09T19:36:36.918Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .FF.F       [ 44%]
Aborted by [8mha:////4D7PK6Qhus1VBgfz6YYbADYkx4cYqY3vMQeQJIB6x//WAAAAmh+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAypEgY+/dLi1CL95IzE3Mz0tCIAIaPy6sMAAAA=[0mFrancis Charette-Migneault
[2024-01-09T19:37:09.292Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb .FF..FFFFFFFF.Sending interrupt signal to process
[2024-01-09T19:37:10.441Z] Terminated
[2024-01-09T19:37:10.449Z] script returned exit code 143
[8mha:////4ItaOhsIRFfcNciVdA3A5uvEe7JeZm9hBtUZEY4X9BpjAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIi0SCEqNJabniBiY1xYt0Z2yGpeBFf4w8EIlGxxWpnm3m+YJkiHDlabA11jlLjMPjeTgsHjt3F84Atn7FhSuwNSjNI1qaeUXI2MKcoYSFgZajxnBzZDGvRqruqvCJbnXKcvoOA0ukbPKAQkzmrmL8wRthce52Yfp6/AhhDyFBu95/avQGMKkS1vgAAAA==[0m[Pipeline] }
[8mha:////4Aio6UelSH6cmkJHznIJktbtMlo1XW0Qqw/RvIKKjzc1AAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOKCQKJBqWgtN7zAJMY4se6C7eBUvIiv8QcsIlGxxWpnm3m9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKZhiFDut7l2hw9mQDtDvgAAAA==[0m[Pipeline] // withCredentials
[8mha:////4FQEhWYsKsWgvD3Y8RE20WPjUSxuhfFY+TD3rc1qM7IiAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiQFQoFa3lhheYxBgn1l2wHZyKF/E1/oBFJCq2WO1sM683LIOHmr3BTlNvKTQWBzeavDCx76+OE3Z8wYYpsNModZLc6tOMkqOGOUUJCwErTY3jYMlEWItOPVTlFJnqHH3+jgJK297hCYXI5qh8/MLkYXMb28D08/wVwDQMEcr9Ntfu8AF5wm4RvgAAAA==[0m[Pipeline] }
[8mha:////4KNTodaWsetSNNWqE//y+IOyoGMxsdvVSBLk6WKj+MzbAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiBTpEldZKwwtMbIwT687YDknFi/gaf8AiEhVbrHa2mdcb1jHAiYPBXtNgKXYWvRtNXjhxGK6OJ+z5gh1TZKex1VPLSjcLtpw0LClKWAnYaOocR0smwVb08iErJ8lU5xTydxRQWnWHJxQim5MM6QtzgN1tVJHp5/krgNn7BOW+zlUfPkGA/eC+AAAA[0m[Pipeline] // script
[8mha:////4KR6tQZJrWFxN0vyZP8vuQmvK2rFt9tOj0oriz2EfOnHAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPMKRSImoaC03vMAkh3Fi3Rnbwal4EV/jD1hEomKL1c4283rDOgY4cjBiQBotxc4K7yZTlsgcxqvjLAa+iI4pskOhMCvu8bSg4oSwpKphJWGD1DmOlkyCrRz0QzdOk2nOKZTvIKG2/R2eUMliTjqkL8wBdrepj0w/z18BzN4nqNu21L79AEfGxbW+AAAA[0m[Pipeline] }
[8mha:////4F01XPIVLu7JBIaS3rjBN4HkNuOk+B1MGW9P2m5t4llYAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiBTpEldZKwwtMbIwT687YDknFi/gaf8AiEhVbrHa2mdcb1jHAiYPBXtNgKXYWvRtNXjhxGK6OJ+z5gh1TZKex1VPLSjcLtpw0LClKWAnYaOocR0smwVb08iErJ8lU5xTydxRQWnWHJxQim5MM6QtzgN1tVJHp5/krgNn7BOX+kKuuP2nGx9++AAAA[0m[Pipeline] // stage
[8mha:////4DuQWrwdSKxqd4Ws+o38dPis/OkEWi6lu/G6AZs3CpBnAAAAqB+LCAAAAAAAAP9tjTEOwjAUQ3+LOrAycohUgJhQJ9aoCycITQhJo/9LkpJOnIircQcClZjwYNmWrPd8QRU8NOQ1swp7g6EzbHCjzokl8v3FUWKWzqwjDOQUa1VqSarjXFuKCmYVJSw4LBV2joJBHWHFrbiL2gnU9Sn6vB04lEbe4AEFz+QofPyWycP6OspA+OP8BcA0RCi3m2y7/edZvQEO/65mwgAAAA==[0m[Pipeline] stage
[8mha:////4N4We8ek5/h9OvNnQ1lyXKfM0ybgoVYvsT4MDoMzUNyTAAAApx+LCAAAAAAAAP9tjTEOwjAUQ3+LOrAycoh0QLAgpq5RFk4QmhDSRv+Xn5R24kRcjTvQUokJD5ZtyXqvNxSR4UTsRGOx9RhrL7rQuymJgbi9BhpEQxdRE0YKVig7KDK2WqqiZGFRlsNKwtpiHSh6dAk2stEPXQaNrjwnnrajhNybOzwhkxM5aU7fMjJsb72JhD/OXwCMXYJ8t5/tMD+LD02yRsvCAAAA[0m[Pipeline] { (Declarative: Post Actions)
[8mha:////4NSN38fl2W391nE1GYw8ukcOn4eLq1zoMVEI1VLYRceiAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOJSIEGBqNJabniBiY1xYt0Z2yGpeBFf4w8EIlGxxWpnm3m+YJkiHDlabA11jlLjMPjeTgsHjt3F84Atn7FhSuwNSjNI1qaeUXI2MKcoYSFgZajxnBzZDGvRqruqvCJbnXKcvoOA0ukbPKAQkzmrmL8wRthce52Yfp6/AhhDhnK7+9Q+vAF9GxrMvgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:10.845Z] Archiving artifacts
[2024-01-09T19:37:10.870Z] Recording fingerprints
[8mha:////4ILGmJge/9XDdI8sXXOB4l4xNVkY2uXECYRIerEjIw8oAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOJSICEkRJXWcsMLTGyME+vO2A5JxYv4Gn8gEImKLVY728zzBcsU4cjRYmuoc5Qah8H3dlo4cOwungds+YwNU2JvUJpBsjb1jJKzgTlFCQsBK0ON5+TIZliLVt1V5RXZ6pTj9B0ElE7f4AGFmMxZxfyFMcLm2uvE9PP8FcAYMpTb3af24Q2yB4JLvgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:10.936Z] Archiving artifacts
[2024-01-09T19:37:12.948Z] Recording fingerprints
[8mha:////4IF5gHMW7IfqA28qOW3Ob2IYZOQWeHvBLJTakZpvnp51AAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOJSICEhRJXWcsMLTGyME+vO2A5JxYv4Gn8gEImKLVY728zzBcsU4cjRYmuoc5Qah8H3dlo4cOwungds+YwNU2JvUJpBsjb1jJKzgTlFCQsBK0ON5+TIZliLVt1V5RXZ6pTj9B0ElE7f4AGFmMxZxfyFMcLm2uvE9PP8FcAYMpTb3af24Q3zNplSvgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:13.228Z] Archiving artifacts
[2024-01-09T19:37:13.274Z] Recording fingerprints
[8mha:////4MwiYGSbSZDrDt6rb7AjDZgJrIpoAZj8jdfzkkZu3Y6TAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOKCBKJBqWgtN7zAJMY4se6C7eBUvIiv8QcsIlGxxWpnm3m9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKYhQrk75Npvhw8/mxSGvgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:13.327Z] Archiving artifacts
[2024-01-09T19:37:13.358Z] Recording fingerprints
[8mha:////4ChQU0L8b7yzNBK4Qecbahzv6Fki7OHgcCGOzh+FMG2nAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiBKJBqWgtN7zAJMY4se6C7eBUvIiv8QcsIlGxxWpnm3m9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKYhQrk75Npvhw9+qg+fvgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:13.428Z] Archiving artifacts
[2024-01-09T19:37:13.465Z] Recording fingerprints
[8mha:////4COiEyXTFMQ9+Rm5mJN/AGgyv2+rMMUSUIOmHsB3w2J+AAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiAaJBqWgtN7zAJMY4se6C7eBUvIiv8QcsIlGxxWpnm3m9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKYhQrk75Npvhw+9+SK0vgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:13.524Z] Archiving artifacts
[2024-01-09T19:37:13.601Z] Recording fingerprints
[8mha:////4J3gLKSOu0/gozlxvjLGfIzNL6Pbiuk1VK63hpZd8S9zAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOJSBNEgqrRWGl5gYmOcWHfGdkgqXsTX+AMWkajYYrWzzbzesI4BThwM9poGS7Gz6N1o8sKJw3B1PGHPF+yYIjuNrZ5aVrpZsOWkYUlRwkrARlPnOFoyCbailw9ZOUmmOqeQv6OA0qo7PKEQ2ZxkSF+YA+xuo4pMP89fAcw+QVkfcu1r/wH8yDmtvgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:14.102Z] Archiving artifacts
[2024-01-09T19:37:14.141Z] Recording fingerprints
[8mha:////4G5uIRbkJeaig52MLfD9xv8Nb7kfLaqFinRw+RlJonU2AAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPMIpiGgQVVorDS8w8WGcWHfGdkgqXsTX+AMWkajYYrWzzbzesI4BThyM6JEGS7GzwrvR5CUmDsPV8SR6voiOKbJD0eLUssZmwZYTwpKihJWEDVLnOFoyCbayVw9VOUWmOqeQv6OE0uo7PKGQ2ZxUSF+YA+xuo45MP89fAcw+Qbk/5Kpr/wE7XnjivgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:14.218Z] Archiving artifacts
[2024-01-09T19:37:14.279Z] Recording fingerprints
[8mha:////4Mvo1BkYPEme5kaGYwfMPGdAn3h6z1MZTQvQAXQpd3AkAAAApR+LCAAAAAAAAP9tjTsOwjAQRDdBFLSUHGJT8GlQKlrLDScwiTFOrN1gOzgVJ+Jq3AGLSFRMMZo3zXu9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKYhQrk95Nrthw96b2P7vgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:14.444Z] Archiving artifacts
[2024-01-09T19:37:14.907Z] Recording fingerprints
[8mha:////4BxJcu6Mhf32TKJrQUvGOWd3sHkWoOwAUOJvn78NS6VSAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOJSgNIgqrRWGl5gYmOcWHfGdkgqXsTX+AMWkajYYrWzzbzesI4BThwM9poGS7Gz6N1o8sKJw3B1PGHPF+yYIjuNrZ5aVrpZsOWkYUlRwkrARlPnOFoyCbailw9ZOUmmOqeQv6OA0qo7PKEQ2ZxkSF+YA+xuo4pMP89fAcw+Qbmvcx1q/wG5PE7QvgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:15.094Z] Archiving artifacts
[8mha:////4GO1B9xqDMc3Y88TWfU6Yi2kkOsY1jMmzauCbLhAJwyKAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOJSgKBAqWgtN7zAJMY4se6C7eBUvIiv8QcsIlGxxWpnm3m9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKYhQrnd59odhg/4DVXJvgAAAA==[0m[Pipeline] archiveArtifacts
[2024-01-09T19:37:15.155Z] Archiving artifacts
[8mha:////4NPt9oC5FBsaixMpbG2Vgr+pYl8XGjukBsva8CWoEC0LAAAAph+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOJSgBASSkVrueEFJjHGiXUXbAen4kV8jT9gEYmKLVY728zrDcvgoWZvsNPUWwqNxcGNJi9M7Pur44QdX7BhCuw0Sp0kt/o0o+SoYU5RwkLASlPjOFgyEdaiUw9VOUWmOkefv6OA0rZ3eEIhsjkqH78wedjcxjYw/Tx/BTANEcrtPtfuMHwANxHNTr4AAAA=[0m[Pipeline] emailextrecipients
[8mha:////4JBwrFm/0BDxJd28CTylmcE02xhjnDQaki1Jq/4AEW2eAAAAph+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOJSgJAQSkVrueEFJjHGiXUXbAen4kV8jT9gEYmKLVY728zrDcvgoWZvsNPUWwqNxcGNJi9M7Pur44QdX7BhCuw0Sp0kt/o0o+SoYU5RwkLASlPjOFgyEdaiUw9VOUWmOkefv6OA0rZ3eEIhsjkqH78wedjcxjYw/Tx/BTANEcrtPtfuMHwAdiDWV74AAAA=[0m[Pipeline] step
[8mha:////4KojkB8TKn7pKbfz2v9XOrtuJMHB5JjbAIzrHIHirXQOAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOKChKBBqWgtN7zAJMY4se6C7eBUvIiv8QcsIlGxxWpnm3m9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKZhiFDut7l2hw8dRPA5vgAAAA==[0m[Pipeline] }
[8mha:////4FScbrhKv8Si2MklnVibpA4A1BxVHfhbvTRVuDRpDSaUAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIihFIhqrRWGl5gYmOcWHfGdkgqXsTX+AMWkajYYrWzzbzesI4BThwM9poGS7Gz6N1o8sKJw3B1PGHPF+yYIjuNrZ5aVrpZsOWkYUlRwkrARlPnOFoyCbailw9ZOUmmOqeQv6OA0qo7PKEQ2ZxkSF+YA+xuo4pMP89fAczeJyjrfa5D/QECxqVrvgAAAA==[0m[Pipeline] // stage
[8mha:////4Fw6vIGI2xNAaETf5rX3RXhuvh/PDdAzPQzPyDrn4GsMAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPMIRkagQVVorDS8w8WGcWHfGdkgqXsTX+AMWkajYYrWzzbzesI4BThyM6JEGS7GzwrvR5CUmDsPV8SR6voiOKbJD0eLUssZmwZYTwpKihJWEDVLnOFoyCbayVw9VOUWmOqeQv6OE0uo7PKGQ2ZxUSF+YA+xuo45MP89fAczeJygPda56/wH0QUfzvgAAAA==[0m[Pipeline] }
[8mha:////4Gx8ZK4G7s3szfZjQe4yyQGDULl8r/MxuiCfHgkt+hhTAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOKCQFQoFa3lhheYxBgn1l2wHZyKF/E1/oBFJCq2WO1sM683LIOHmr3BTlNvKTQWBzeavDCx76+OE3Z8wYYpsNModZLc6tOMkqOGOUUJCwErTY3jYMlEWItOPVTlFJnqHH3+jgJK297hCYXI5qh8/MLkYXMb28D08/wVwDQMEcrDPtdu+wHHohxPvgAAAA==[0m[Pipeline] // timeout
[8mha:////4Dr1Bd9XnpD7W4ukUv4MR2OdYWV2WSasrALHdGy56iWmAAAAph+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiJCgQSkVrueEFJjHGiXUXbAen4kV8jT9gEYmKLVY728zrDcvgoWZvsNPUWwqNxcGNJi9M7Pur44QdX7BhCuw0Sp0kt/o0o+SoYU5RwkLASlPjOFgyEdaiUw9VOUWmOkefv6OA0rZ3eEIhsjkqH78wedjcxjYw/Tx/BTANQ4Ryv8u1PXwAGHnoAL4AAAA=[0m[Pipeline] }
[8mha:////4EQfO6o7JkCpkeoEdltFIcIqnCpHTJVKIJJ6lUtPuw+wAAAAph+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPMIRRSQkRJXWSsMLTHwYJ9adsR2SihfxNf6ARSQqtljtbDOvN6xjgBMHI3qkwVLsrPBuNHmJicNwdTyJni+iY4rsULQ4tayxWbDlhLCkKGElYYPUOY6WTIKt7NVDVU6Rqc4p5O8oobT6Dk8oZDYnFdIX5gC726gj08/zVwCz9wnKus61P3wAK5qzvL4AAAA=[0m[Pipeline] // timestamps
[8mha:////4DaXclT9PDXKywCC8y0Reoo4mYsWVA5/co48+DE0wAP9AAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiCqBAqWgtN7zAJMY4se6C7eBUvIiv8QcsIlGxxWpnm3m9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKZhiFDu9rm2hw8U9ZiqvgAAAA==[0m[Pipeline] }
[8mha:////4EFb2CpPyfZyYfXSY4/e/0MhVJmkeiKTTRaJHaHIAuyZAAAAqx+LCAAAAAAAAP9b85aBtbiIQSujNKU4P0+vIKc0PTOvWC8xrzgzOT8nv0gvMbkkEyjhCKb88ktSdwVsu2efuqidiYHRh4EDIu2ZUsIg5JOVWJaon5OYl64fXFKUmZduXVHEIAU1ODk/rzg/J1XPGUKDDGKAAEYmBoaKghIGFfM0Q4ukRDMD3RRLS0Ndk6QUE12LVIs0XcNEk2TzVLNEsySzJABIXmJmrwAAAA==[0m
[8mha:////4OvVjBUfHjGh+usvQ124ua0x4itVhM/lWD3Qk7cNzI1jAAAApR+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiCqBAqWgtN7zAJMY4se6C7eBUvIiv8QcsIlGxxWpnm3m9YRk81OwNdpp6S6GxOLjR5IWJfX91nLDjCzZMgZ1GqZPkVp9mlBw1zClKWAhYaWocB0smwlp06qEqp8hU5+jzdxRQ2vYOTyhENkfl4xcmD5vb2Aamn+evAKZhiFDuDrm2+w8nFsMWvgAAAA==[0m[Pipeline] // ansiColor
[8mha:////4KLU+6Z1ffGhQMfDsCrAppAB96nkob77cEvF/QNY2LT0AAAAph+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPMIRRSQkRJXWSsMLTHwYJ9adsR2SihfxNf6ARSQqtljtbDOvN6xjgBMHI3qkwVLsrPBuNHmJicNwdTyJni+iY4rsULQ4tayxWbDlhLCkKGElYYPUOY6WTIKt7NVDVU6Rqc4p5O8oobT6Dk8oZDYnFdIX5gC726gj08/zVwCz9wnK+pBrX38ASPWcfr4AAAA=[0m[Pipeline] }
$ docker stop --time=1 12edc6add5702895391e9ca5f84197f3f17088b2f4b72a3b2c834b5cbfb3180d
$ docker rm -f 12edc6add5702895391e9ca5f84197f3f17088b2f4b72a3b2c834b5cbfb3180d
[8mha:////4AIFJhuGUX/Uty8x99a5AywAvS9PE7zs7wekaYpXupFnAAAAph+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPOIiJCgQSkVrueEFJjHGiXUXbAen4kV8jT9gEYmKLVY728zrDcvgoWZvsNPUWwqNxcGNJi9M7Pur44QdX7BhCuw0Sp0kt/o0o+SoYU5RwkLASlPjOFgyEdaiUw9VOUWmOkefv6OA0rZ3eEIhsjkqH78wedjcxjYw/Tx/BTANQ4Ryf8i13X0AexbHwr4AAAA=[0m[Pipeline] // withDockerContainer
[8mha:////4Pi9XiC/IW6ldszwhEFPEofb43Ju6A5sNVsYOyfhy4LKAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPMIhDQ2iSmul4QUmPowT687YDknFi/gaf8AiEhVbrHa2mdcb1jHAiYMRPdJgKXZWeDeavMTEYbg6nkTPF9ExRXYoWpxa1tgs2HJCWFKUsJKwQeocR0smwVb26qEqp8hU5xTyd5RQWn2HJxQym5MK6QtzgN1t1JHp5/krgNn7BOVhn6uuP6GpSqS+AAAA[0m[Pipeline] }
[8mha:////4JnjGFeippPettZx1BoF2QESywYuXDPhgPvND/+094JdAAAApB+LCAAAAAAAAP9tjTEOwjAQBC9BFLSUPMIhDQ2iSmul4QUmPowT687YDknFi/gaf8AiEhVbrHa2mdcb1jHAiYMRPdJgKXZWeDeavMTEYbg6nkTPF9ExRXYoWpxa1tgs2HJCWFKUsJKwQeocR0smwVb26qEqp8hU5xTyd5RQWn2HJxQym5MK6QtzgN1t1JHp5/krgNn7BOWhzlXvP5JKERi+AAAA[0m[Pipeline] // withEnv
[8mha:////4H6iMnIYjumu76MMuA9fPoCq/vMVTYeeBr/m1bYoU32PAAAApB+LCAAAAAAAAP9tjTEOwjAQBDdBFLSUPMKREKJBVLRWGl5gEmOcWHfBvpBUvIiv8QciIlGx1c4083pjmSKOHJ1qLLWeUuVVF3o3PTVwbK+BB9XwRVVMiYNVpR1Kru1pxpLFYl6WY6GxslQFTp6cYK0b8zBFMOSKs8TJHTRyX9/xRKanspgoXxgjNre+Tky/zt8Axq4T5PutINt9AF1IgLW9AAAA[0m[Pipeline] }
[8mha:////4OI4KkmzI8k/JgzG2qbaQXde9QSao2WpKFAcpcxTCG5sAAAAox+LCAAAAAAAAP9tjbEOgjAURS8YB1dHP6IMJC7GybVh8Qsq1Fpo3sP2IUx+kb/mP0gkcfJO95zlvN5Yp4gjR6daS52nVHvVh8HNT40cu2vgUbV8UTVT4mBVZceKG3tasGKxWJblWGlsLNWBkycn2OrWPEwRDLniLHF2B43cN3c8kem5LCbKF6aI3W1oEtOv8zeAqe8F+b4UZOUHTvSEFr0AAAA=[0m[Pipeline] // node
[8mha:////4PNM6dqYnbJycYQsgBeaYSGVlaYAXkcEjqCxpB59pS4vAAAApB+LCAAAAAAAAP9tjTEOwjAQBDdBFLSUPMKREKJBVLRWGl5gEmOcWHfBvpBUvIiv8QciIlGx1c4083pjmSKOHJ1qLLWeUuVVF3o3PTVwbK+BB9XwRVVMiYNVpR1Kru1pxpLFYl6WY6GxslQFTp6cYK0b8zBFMOSKs8TJHTRyX9/xRKanspgoXxgjNre+Tky/zt8Axq4T5PudINt+AMgYo9O9AAAA[0m[Pipeline] End of Pipeline
Finished: ABORTED

crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2403/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1488/

NOTEBOOK TEST RESULTS

    
[2024-01-09T19:48:58.852Z] ============================= test session starts ==============================
[2024-01-09T19:48:58.852Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-09T19:48:58.852Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-09T19:48:58.852Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-09T19:48:58.852Z] collected 265 items
[2024-01-09T19:48:58.852Z] 
[2024-01-09T19:49:09.397Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-09T19:49:42.836Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-09T19:49:48.737Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-09T19:49:57.408Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-09T19:50:07.476Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-09T19:50:10.775Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 22%]
[2024-01-09T19:57:34.203Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-09T19:57:34.203Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-09T19:57:46.318Z] ...............                                                          [ 33%]
[2024-01-09T19:57:56.276Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-09T19:58:03.590Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-09T19:58:19.627Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-09T19:58:21.267Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 40%]
[2024-01-09T19:58:26.589Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-09T19:58:31.284Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-09T20:02:48.094Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-09T20:04:03.573Z] .............                                                            [ 55%]
[2024-01-09T20:04:05.749Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-01-09T20:04:07.893Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-09T20:04:24.457Z] .................                                                        [ 66%]
[2024-01-09T20:04:33.075Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-09T20:04:34.466Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-09T20:04:52.269Z] .........                                                                [ 72%]
[2024-01-09T20:05:01.835Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-09T20:05:11.073Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-09T20:05:12.456Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-09T20:05:15.550Z] ......                                                                   [ 81%]
[2024-01-09T20:05:23.685Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-09T20:05:39.690Z] .............                                                            [ 86%]
[2024-01-09T20:05:49.718Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-09T20:06:26.838Z] ....s.                                                                   [ 89%]
[2024-01-09T20:06:34.989Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-09T20:06:50.530Z] ...                                                                      [ 90%]
[2024-01-09T20:07:05.457Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-09T20:07:28.381Z] ......                                                                   [ 93%]
[2024-01-09T20:07:30.022Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-09T20:10:10.097Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-09T20:10:10.097Z] 
[2024-01-09T20:10:10.097Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2406/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1490/

NOTEBOOK TEST RESULTS

    
[2024-01-11T19:33:57.379Z] ============================= test session starts ==============================
[2024-01-11T19:33:57.379Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-11T19:33:57.379Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-11T19:33:57.379Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-11T19:33:57.379Z] collected 265 items
[2024-01-11T19:33:57.379Z] 
[2024-01-11T19:34:09.407Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-11T19:34:49.674Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-11T19:34:53.503Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-11T19:35:02.160Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-11T19:35:12.519Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-11T19:35:18.837Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ......FF       [ 22%]
[2024-01-11T19:42:51.989Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-11T19:42:51.989Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-11T19:42:53.040Z] ...............                                                          [ 33%]
[2024-01-11T19:43:02.409Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-11T19:43:10.940Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-11T19:43:28.566Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-11T19:43:29.952Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 40%]
[2024-01-11T19:43:35.368Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-11T19:43:39.878Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-11T19:47:43.199Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-11T19:49:00.230Z] .............                                                            [ 55%]
[2024-01-11T19:49:03.604Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-01-11T19:49:05.830Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-11T19:49:24.087Z] .................                                                        [ 66%]
[2024-01-11T19:49:31.803Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-11T19:49:33.196Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-11T19:50:11.139Z] .........                                                                [ 72%]
[2024-01-11T19:50:22.270Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-11T19:50:32.022Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-11T19:50:33.408Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-11T19:50:36.796Z] ......                                                                   [ 81%]
[2024-01-11T19:50:51.708Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-11T19:51:10.660Z] .............                                                            [ 86%]
[2024-01-11T19:51:20.701Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-11T19:52:04.283Z] ....s.                                                                   [ 89%]
[2024-01-11T19:52:12.502Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-11T19:52:25.804Z] ...                                                                      [ 90%]
[2024-01-11T19:52:40.721Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-11T19:53:04.606Z] ......                                                                   [ 93%]
[2024-01-11T19:53:06.271Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-11T19:55:40.434Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-11T19:55:40.434Z] 
[2024-01-11T19:55:40.434Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2408/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1492/

NOTEBOOK TEST RESULTS

    
[2024-01-12T23:15:44.708Z] ============================= test session starts ==============================
[2024-01-12T23:15:44.708Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-12T23:15:44.708Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-12T23:15:44.708Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-12T23:15:44.708Z] collected 265 items
[2024-01-12T23:15:44.708Z] 
[2024-01-12T23:15:56.886Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-12T23:16:37.204Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-12T23:16:41.875Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-12T23:16:51.511Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-12T23:17:01.579Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-12T23:17:08.791Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ......FF       [ 22%]
[2024-01-12T23:24:55.255Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-12T23:24:55.256Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-12T23:24:59.421Z] ...............                                                          [ 33%]
[2024-01-12T23:25:09.140Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-12T23:25:18.316Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-12T23:25:36.469Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-12T23:25:37.855Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 40%]
[2024-01-12T23:25:43.097Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-12T23:25:47.531Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-12T23:30:36.167Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-12T23:31:48.382Z] .............                                                            [ 55%]
[2024-01-12T23:31:52.581Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-01-12T23:31:55.165Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-12T23:32:11.642Z] .................                                                        [ 66%]
[2024-01-12T23:32:19.785Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-12T23:32:21.169Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-12T23:32:39.529Z] .........                                                                [ 72%]
[2024-01-12T23:32:48.531Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-12T23:32:57.933Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-12T23:32:59.582Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-12T23:33:02.962Z] ......                                                                   [ 81%]
[2024-01-12T23:33:12.960Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-12T23:33:30.994Z] .............                                                            [ 86%]
[2024-01-12T23:33:41.088Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-12T23:34:20.152Z] ....s.                                                                   [ 89%]
[2024-01-12T23:34:28.297Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-12T23:34:43.903Z] ...                                                                      [ 90%]
[2024-01-12T23:34:56.149Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-12T23:35:18.555Z] ......                                                                   [ 93%]
[2024-01-12T23:35:21.321Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-12T23:37:55.496Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-12T23:37:55.496Z] 
[2024-01-12T23:37:55.496Z] =================================== FAILURES ===================================

fmigneault commented 10 months ago

@tlvu Friendly reminder for remaining discussions.

crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2425/
Result : success

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-69.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1501/

NOTEBOOK TEST RESULTS

    
[2024-01-19T19:13:55.760Z] ============================= test session starts ==============================
[2024-01-19T19:13:55.760Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-19T19:13:55.760Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-19T19:13:55.760Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-19T19:13:55.760Z] collected 264 items
[2024-01-19T19:13:55.760Z] 
[2024-01-19T19:14:07.019Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-19T19:14:47.308Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-19T19:14:51.812Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-19T19:15:00.795Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-19T19:15:10.900Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-19T19:15:20.654Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-01-19T19:22:39.220Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-19T19:22:39.481Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-19T19:22:46.983Z] ...............                                                          [ 33%]
[2024-01-19T19:22:56.627Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-19T19:23:03.340Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-19T19:23:19.970Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-19T19:23:25.198Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-19T19:23:29.874Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-19T19:28:12.860Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-19T19:29:25.613Z] .............                                                            [ 54%]
[2024-01-19T19:29:29.699Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-01-19T19:29:32.037Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-19T19:29:48.690Z] .................                                                        [ 65%]
[2024-01-19T19:29:56.851Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-19T19:29:58.237Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-19T19:30:16.855Z] .........                                                                [ 71%]
[2024-01-19T19:30:26.363Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-19T19:30:35.419Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-19T19:30:36.815Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-19T19:30:39.891Z] ......                                                                   [ 81%]
[2024-01-19T19:30:46.484Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-19T19:31:01.826Z] .............                                                            [ 86%]
[2024-01-19T19:31:11.832Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-19T19:31:48.419Z] ....s.                                                                   [ 89%]
[2024-01-19T19:31:56.570Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-19T19:32:10.725Z] ...                                                                      [ 90%]
[2024-01-19T19:32:22.972Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-19T19:32:45.067Z] ......                                                                   [ 93%]
[2024-01-19T19:32:47.292Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-19T19:35:26.928Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-19T19:35:26.928Z] 
[2024-01-19T19:35:26.928Z] ================= 263 passed, 1 skipped in 1290.27s (0:21:30) ==================

tlvu commented 10 months ago

@tlvu Friendly reminder for remaining discussions.

Sorry I got caught up in other issues since back from the holidays. Will take a look at this next week.

tlvu commented 10 months ago

I self request-review again so if shows up again on my list of PR to review.

fmigneault commented 10 months ago

run tests

crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2438/
Result : success

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1505/

NOTEBOOK TEST RESULTS

    
[2024-01-23T19:06:54.919Z] ============================= test session starts ==============================
[2024-01-23T19:06:54.919Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-23T19:06:54.919Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-23T19:06:54.919Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-23T19:06:54.919Z] collected 264 items
[2024-01-23T19:06:54.919Z] 
[2024-01-23T19:07:06.293Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-23T19:07:46.531Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-23T19:07:50.769Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-23T19:08:00.001Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-23T19:08:10.081Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-23T19:08:18.929Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-01-23T19:15:55.041Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-23T19:15:55.041Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-23T19:15:56.405Z] ...............                                                          [ 33%]
[2024-01-23T19:16:08.575Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-23T19:16:15.498Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-23T19:16:31.689Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-23T19:16:36.725Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-23T19:16:40.723Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-23T19:21:15.594Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-23T19:22:30.791Z] .............                                                            [ 54%]
[2024-01-23T19:22:32.702Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-01-23T19:22:34.929Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-23T19:22:52.925Z] .................                                                        [ 65%]
[2024-01-23T19:23:00.833Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-23T19:23:02.218Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-23T19:23:20.858Z] .........                                                                [ 71%]
[2024-01-23T19:23:30.578Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-23T19:23:39.875Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-23T19:23:41.793Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-23T19:23:44.860Z] ......                                                                   [ 81%]
[2024-01-23T19:23:54.851Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-23T19:24:11.304Z] .............                                                            [ 86%]
[2024-01-23T19:24:21.316Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-23T19:25:01.543Z] ....s.                                                                   [ 89%]
[2024-01-23T19:25:09.679Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-23T19:25:24.439Z] ...                                                                      [ 90%]
[2024-01-23T19:25:39.358Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-23T19:26:03.225Z] ......                                                                   [ 93%]
[2024-01-23T19:26:05.435Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-23T19:28:39.849Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-23T19:28:39.850Z] 
[2024-01-23T19:28:39.850Z] ================= 263 passed, 1 skipped in 1301.41s (0:21:41) ==================

crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2441/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1506/

NOTEBOOK TEST RESULTS

    
[2024-01-25T16:11:15.254Z] ============================= test session starts ==============================
[2024-01-25T16:11:15.254Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-25T16:11:15.254Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-25T16:11:15.254Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-25T16:11:15.254Z] collected 264 items
[2024-01-25T16:11:15.254Z] 
[2024-01-25T16:11:25.998Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-25T16:12:06.242Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-25T16:12:09.788Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-25T16:12:19.410Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-25T16:12:29.138Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-25T16:12:35.827Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-01-25T16:20:21.918Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-25T16:20:21.918Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-25T16:20:23.866Z] ...............                                                          [ 33%]
[2024-01-25T16:20:33.150Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-25T16:20:39.830Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-25T16:20:55.750Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-25T16:21:00.716Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-25T16:21:04.951Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-25T16:26:17.605Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-25T16:27:33.578Z] .............                                                            [ 54%]
[2024-01-25T16:27:36.015Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-01-25T16:27:38.245Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-25T16:27:54.553Z] .................                                                        [ 65%]
[2024-01-25T16:28:03.207Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-25T16:28:04.608Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-25T16:29:17.888Z] ......FFF                                                                [ 71%]
[2024-01-25T16:29:28.201Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-25T16:29:37.792Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-25T16:29:39.175Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-25T16:29:42.573Z] ......                                                                   [ 81%]
[2024-01-25T16:29:50.712Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-25T16:30:08.570Z] .............                                                            [ 86%]
[2024-01-25T16:30:20.804Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-25T16:30:58.202Z] ....s.                                                                   [ 89%]
[2024-01-25T16:31:06.362Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-25T16:31:21.904Z] ...                                                                      [ 90%]
[2024-01-25T16:31:34.143Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-25T16:31:55.838Z] ......                                                                   [ 93%]
[2024-01-25T16:31:58.590Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-25T16:34:38.189Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-25T16:34:38.189Z] 
[2024-01-25T16:34:38.189Z] =================================== FAILURES ===================================

fmigneault commented 10 months ago

run tests

note: Since merge of 24950ba, the flag to skip CI tests is embedded in the PR page. They will not run automatically.

mishaschwartz commented 10 months ago

Since merge of 24950ba, the flag to skip CI tests is embedded in the PR page.

Oh because of the comment? That's strange behaviour. Ok good to know for next time

fmigneault commented 10 months ago

Oh because of the comment

Yes. I was surprised to see that as well. I pushed an update to use the GitHub API to extract only the PR description in the future. Not sure if it will apply immediately for this PR though.

crim-jenkins-bot commented 9 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2481/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : branch_name
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://

Infrastructure deployment failed. Instance has not been destroyed. @matprov

mishaschwartz commented 9 months ago

@tlvu did you have more tests you wanted to run with this or can we move forward with this PR?

tlvu commented 9 months ago

@tlvu did you have more tests you wanted to run with this or can we move forward with this PR?

@mishaschwartz

Apologies for the long delay. I was deploying this PR on my test VM and since this include the 2.0.0 version with Cowbird, I was not able to get my Jupyter working with Cowbird.

For context, my VM is an existing deployment with existing Jupyter users and their data, simulating the production. So Cowbird was conflicting with our setup because we also had our home grown ad-hoc sharing we would like to keep, see https://github.com/bird-house/birdhouse-deploy/blob/67c6ca1d22c47d9bdf6f6e239f808ef3ec9af0bb/birdhouse/env.local.example#L386-L426 and also Cowbird assume a different user workspace with the apparent effect that existing data for the user has disappeared !

Anyways, so it seems Cowbird activation has been tested for fresh new system but not for existing system with existing data and existing users. It's basically missing migration documentation and backward compatibility, which I am discovering with pain and bruises.

Not sure how much more road block I will encounter.

If you are in need for this PR, I guess it can be merged and we can open new PR once I have actually deployed it and test it, just like Cowbird right now.

However, not all the issues I have already found has been fixed in this PR yet. I can approuve once those are fixed.

mishaschwartz commented 9 months ago

@tlvu Is the issue you're encountering the same one that is described in #392?

If so, I'm not sure that that issue is related to this PR. We definitely need to resolve that issue but I don't think that it should block this PR in the meantime.

However, not all the issues I have already found has been fixed in this PR yet. I can approuve once those are fixed.

That's fine, @fmigneault do you have time to have another look at this PR in the near future? If you want I can also take over to make the last few updates that @tlvu requested if you don't have time.

If you are in need for this PR

I'd love to get this finalized soon so that I can continue working on resolving #414

fmigneault commented 9 months ago

run tests

crim-jenkins-bot commented 9 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2484/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : branch_name
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://

Infrastructure deployment failed. Instance has not been destroyed. @matprov

fmigneault commented 9 months ago

@tlvu @mishaschwartz What was left to do? I thought I had pushed every update to address the comments.

fmigneault commented 9 months ago

run tests

crim-jenkins-bot commented 9 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2485/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1524/

NOTEBOOK TEST RESULTS

    
[2024-02-17T00:54:02.251Z] ============================= test session starts ==============================
[2024-02-17T00:54:02.251Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-17T00:54:02.251Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-17T00:54:02.251Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-17T00:54:02.251Z] collected 264 items
[2024-02-17T00:54:02.251Z] 
[2024-02-17T00:54:13.089Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-17T00:54:47.841Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-17T00:54:51.827Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-17T00:55:00.750Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-17T00:55:10.728Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-17T00:55:19.937Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-02-17T01:02:42.110Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-17T01:02:42.110Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-17T01:02:45.662Z] ...............                                                          [ 33%]
[2024-02-17T01:02:55.077Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-17T01:04:07.833Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .....F            [ 37%]
[2024-02-17T01:04:19.308Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-17T01:05:18.863Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-17T01:05:23.378Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-17T01:08:51.159Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-17T01:10:11.476Z] .............                                                            [ 54%]
[2024-02-17T01:10:15.959Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-17T01:10:18.305Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-17T01:10:34.764Z] .................                                                        [ 65%]
[2024-02-17T01:10:42.869Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-17T01:10:44.259Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-17T01:11:08.560Z] .........                                                                [ 71%]
[2024-02-17T01:11:17.472Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-17T01:11:26.850Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-17T01:11:28.233Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-17T01:11:31.799Z] ......                                                                   [ 81%]
[2024-02-17T01:11:39.950Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-17T01:11:55.114Z] .............                                                            [ 86%]
[2024-02-17T01:12:05.127Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-17T01:12:43.961Z] ....s.                                                                   [ 89%]
[2024-02-17T01:12:52.112Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-17T01:13:06.906Z] ...                                                                      [ 90%]
[2024-02-17T01:13:21.829Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-17T01:13:45.201Z] ......                                                                   [ 93%]
[2024-02-17T01:13:47.936Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-17T01:16:33.611Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-17T01:16:33.611Z] 
[2024-02-17T01:16:33.611Z] =================================== FAILURES ===================================

tlvu commented 9 months ago

@tlvu Is the issue you're encountering the same one that is described in #392?

If so, I'm not sure that that issue is related to this PR. We definitely need to resolve that issue but I don't think that it should block this PR in the meantime.

@mishaschwartz Agreed not blocking, that why I was okay to go forward with this, as long as my other feedbacks are handled.

@tlvu @mishaschwartz What was left to do? I thought I had pushed every update to address the comments.

@fmigneault Quickly from memory: read-configs.include.sh should be moved back to its old location for backward compatibility with external scripts and to make it stand out as an entrypoint for external scripts to hook into the "API" of the platform by sourcing read-configs.include.sh.

crim-jenkins-bot commented 9 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2489/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1526/

NOTEBOOK TEST RESULTS

    
[2024-02-20T19:37:01.784Z] ============================= test session starts ==============================
[2024-02-20T19:37:01.784Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-20T19:37:01.784Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-20T19:37:01.784Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-20T19:37:01.784Z] collected 264 items
[2024-02-20T19:37:01.784Z] 
[2024-02-20T19:37:13.650Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-20T19:37:48.381Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-20T19:37:52.547Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-20T19:38:01.219Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-20T19:38:11.215Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-20T19:38:20.503Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 22%]
[2024-02-20T19:45:51.741Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-20T19:45:51.741Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-20T19:45:52.835Z] ...............                                                          [ 33%]
[2024-02-20T19:46:02.156Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-20T19:47:14.753Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .....F            [ 37%]
[2024-02-20T19:47:25.495Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-20T19:47:30.526Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-20T19:47:35.050Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-20T19:51:16.399Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-20T19:52:31.527Z] .............                                                            [ 54%]
[2024-02-20T19:52:33.476Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-20T19:52:35.712Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-20T19:52:52.250Z] .................                                                        [ 65%]
[2024-02-20T19:52:59.246Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-20T19:53:01.154Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-20T19:53:06.403Z] .FFFFFFFF                                                                [ 71%]
[2024-02-20T19:53:17.272Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-20T19:53:27.023Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-20T19:53:28.412Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-20T19:53:31.478Z] ......                                                                   [ 81%]
[2024-02-20T19:53:38.068Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-20T19:53:52.819Z] .............                                                            [ 86%]
[2024-02-20T19:54:02.831Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-20T19:54:37.576Z] ....s.                                                                   [ 89%]
[2024-02-20T19:54:45.719Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-20T19:55:00.512Z] ...                                                                      [ 90%]
[2024-02-20T19:55:12.754Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-20T19:55:34.778Z] ......                                                                   [ 93%]
[2024-02-20T19:55:38.081Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-20T19:58:17.729Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-20T19:58:17.729Z] 
[2024-02-20T19:58:17.729Z] =================================== FAILURES ===================================

fmigneault commented 9 months ago

@tlvu Reverted move of read-configs.include.sh in the spirit of closing this PR once and for all. Let me know if you find any issues after testing. I found some more locations where COMPOSE_DIR was not defined in the same manner. I did another pass of unification of COMPOSE_DIR such that it is used if provided directly, or attempts auto-resolve with THIS_DIR/THIS_FILE usual method.

crim-jenkins-bot commented 9 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2490/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1528/

NOTEBOOK TEST RESULTS

    
[2024-02-21T00:15:32.514Z] ============================= test session starts ==============================
[2024-02-21T00:15:32.514Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-21T00:15:32.514Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-21T00:15:32.514Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-21T00:15:32.514Z] collected 264 items
[2024-02-21T00:15:32.514Z] 
[2024-02-21T00:15:44.347Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-21T00:16:10.593Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-21T00:16:18.711Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-21T00:16:27.991Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-21T00:16:37.961Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-21T00:16:40.600Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 22%]
[2024-02-21T00:29:53.244Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-21T00:29:55.793Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-21T00:30:04.724Z] ...............                                                          [ 33%]
[2024-02-21T00:30:14.617Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-21T00:30:21.624Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-02-21T00:30:39.168Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-21T00:30:44.727Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-21T00:30:49.505Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-21T00:34:03.021Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-21T00:35:23.224Z] .............                                                            [ 54%]
[2024-02-21T00:35:27.675Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-21T00:35:29.892Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-21T00:35:45.210Z] .................                                                        [ 65%]
[2024-02-21T00:35:53.556Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-21T00:35:54.953Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-21T00:36:13.056Z] .........                                                                [ 71%]
[2024-02-21T00:36:22.278Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-21T00:36:31.506Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-21T00:36:32.911Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-21T00:36:36.001Z] ......                                                                   [ 81%]
[2024-02-21T00:36:44.137Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-21T00:36:59.947Z] .............                                                            [ 86%]
[2024-02-21T00:37:09.950Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-21T00:37:55.183Z] ....s.                                                                   [ 89%]
[2024-02-21T00:38:05.173Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-21T00:38:19.951Z] ...                                                                      [ 90%]
[2024-02-21T00:38:34.866Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-21T00:38:57.419Z] ......                                                                   [ 93%]
[2024-02-21T00:38:58.522Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-21T00:41:32.685Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-21T00:41:32.685Z] 
[2024-02-21T00:41:32.685Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 9 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2492/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1529/

NOTEBOOK TEST RESULTS

    
[2024-02-21T17:26:15.288Z] ============================= test session starts ==============================
[2024-02-21T17:26:15.288Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-21T17:26:15.288Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-21T17:26:15.288Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-21T17:26:15.288Z] collected 264 items
[2024-02-21T17:26:15.288Z] 
[2024-02-21T17:26:26.108Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-21T17:27:00.830Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-21T17:27:05.339Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-21T17:27:14.011Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-21T17:27:24.380Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-21T17:27:58.286Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 22%]
[2024-02-21T17:35:31.492Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-21T17:35:31.492Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-21T17:35:35.995Z] ...............                                                          [ 33%]
[2024-02-21T17:35:45.210Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-21T17:35:52.117Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-02-21T17:36:08.936Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-21T17:36:13.970Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-21T17:36:18.478Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-21T17:40:09.043Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-21T17:41:28.592Z] .............                                                            [ 54%]
[2024-02-21T17:41:31.890Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-21T17:41:34.222Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-21T17:41:51.785Z] .................                                                        [ 65%]
[2024-02-21T17:42:00.426Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-21T17:42:01.835Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-21T17:42:20.909Z] .........                                                                [ 71%]
[2024-02-21T17:42:30.384Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-21T17:42:40.154Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-21T17:42:41.537Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-21T17:42:45.101Z] ......                                                                   [ 81%]
[2024-02-21T17:42:53.249Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-21T17:43:07.979Z] .............                                                            [ 86%]
[2024-02-21T17:43:20.221Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-21T17:44:02.321Z] ....s.                                                                   [ 89%]
[2024-02-21T17:44:10.458Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-21T17:44:25.972Z] ...                                                                      [ 90%]
[2024-02-21T17:44:40.862Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-21T17:45:03.743Z] ......                                                                   [ 93%]
[2024-02-21T17:45:05.367Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-21T17:47:54.472Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-21T17:47:54.472Z] 
[2024-02-21T17:47:54.472Z] =================================== FAILURES ===================================

crim-jenkins-bot commented 9 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2493/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : fix-node-details
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-133.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1530/

NOTEBOOK TEST RESULTS

    
[2024-02-21T17:54:15.311Z] ============================= test session starts ==============================
[2024-02-21T17:54:15.311Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-02-21T17:54:15.311Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-02-21T17:54:15.311Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-02-21T17:54:15.311Z] collected 264 items
[2024-02-21T17:54:15.311Z] 
[2024-02-21T17:54:25.392Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-02-21T17:54:51.642Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-02-21T17:54:59.278Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-02-21T17:55:08.224Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-02-21T17:55:16.600Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-02-21T17:55:50.875Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 22%]
[2024-02-21T18:03:21.622Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-02-21T18:03:21.622Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-02-21T18:03:24.846Z] ...............                                                          [ 33%]
[2024-02-21T18:03:34.559Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-02-21T18:03:40.952Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-02-21T18:03:58.101Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-02-21T18:04:03.180Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-02-21T18:04:07.420Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-02-21T18:07:18.309Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-02-21T18:08:26.840Z] .............                                                            [ 54%]
[2024-02-21T18:08:30.219Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 56%]
[2024-02-21T18:08:32.447Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-02-21T18:08:48.919Z] .................                                                        [ 65%]
[2024-02-21T18:08:56.384Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-02-21T18:08:57.780Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-02-21T18:09:25.570Z] .........                                                                [ 71%]
[2024-02-21T18:09:35.449Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-02-21T18:09:44.940Z] finch-master/docs/source/notebooks/finch-usage.ipynb ...F..              [ 78%]
[2024-02-21T18:09:46.325Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-02-21T18:09:49.387Z] ......                                                                   [ 81%]
[2024-02-21T18:09:55.980Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-02-21T18:10:11.396Z] .............                                                            [ 86%]
[2024-02-21T18:10:21.404Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-02-21T18:10:56.105Z] ....s.                                                                   [ 89%]
[2024-02-21T18:11:04.250Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-02-21T18:11:19.004Z] ...                                                                      [ 90%]
[2024-02-21T18:11:33.933Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-02-21T18:11:55.030Z] ......                                                                   [ 93%]
[2024-02-21T18:11:57.514Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-02-21T18:14:32.162Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-02-21T18:14:32.162Z] 
[2024-02-21T18:14:32.162Z] =================================== FAILURES ===================================

fmigneault commented 9 months ago

@tlvu

All good for me. Please perform an autodeploy test in case there are accidental typo errors.

Can you do this test according to what you expect it to accomplish? We do not employ the autodeploy script at CRIM.

tlvu commented 9 months ago

@tlvu

All good for me. Please perform an autodeploy test in case there are accidental typo errors.

Can you do this test according to what you expect it to accomplish? We do not employ the autodeploy script at CRIM.

@fmigneault

I don't think I should be doing this test for you. We have a fully reproducible stack so that we can be independent in our testing. If the stack is not reproducible, then yes you will need to depend on the person with the proper setup to test for you.

Autodeploy test procedure is well documented here https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/components#how-to-test-platform-autodeploy-is-not-broken-by-a-pr

If me or Misha start playing with Magpie or Weaver code, would you want us to ask you to test for us? I don't think you see yourself as our QA agent.

If you make changes to Magpie, I bet you will test that Weaver, Thredds, Geoserver and all the WPS services are still working properly with the new Magpie because Weaver, Thredds, Geoserver and all the WPS services are part of the stack and the goal is for all the built-in parts to work properly together. Autodeploy is also a built-in part of the stack.

Last time, we discovered very late that Weaver post-docker-compose-up script did not work with Autodeploy because it relies on curl which is not present in the autodeploy container. Had autodeploy be considered much earlier during development, we would not have that last minute surprise before Weaver go-live on Ouranos side.

I went ahead and test autodeploy for you anyways because I think we all want this PR to be merged. Good news, everything is fine with autodeploy.

However I see too many WARNING that I think should probably be at DEBUG level instead to avoid drowning real WARNING. This could be a separate PR, not blocking this one.

WARNING:  Optional variable [GITHUB_CLIENT_ID] is not set. Check env.local file.
WARNING:  Optional variable [GITHUB_CLIENT_SECRET] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_USER_REGISTRATION_SUBMISSION_EMAIL_TEMPLATE] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_USER_REGISTRATION_APPROVAL_EMAIL_RECIPIENT] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_USER_REGISTRATION_APPROVAL_EMAIL_TEMPLATE] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_USER_REGISTRATION_APPROVED_EMAIL_TEMPLATE] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_USER_REGISTRATION_DECLINED_EMAIL_TEMPLATE] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_USER_REGISTRATION_NOTIFY_EMAIL_RECIPIENT] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_USER_REGISTRATION_NOTIFY_EMAIL_TEMPLATE] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_GROUP_TERMS_SUBMISSION_EMAIL_TEMPLATE] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_GROUP_TERMS_APPROVED_EMAIL_TEMPLATE] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_SMTP_FROM] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_SMTP_HOST] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_SMTP_PORT] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_SMTP_SSL] is not set. Check env.local file.
WARNING:  Optional variable [MAGPIE_SMTP_PASSWORD] is not set. Check env.local file.
WARNING:  Optional variable [SECURE_DATA_PROXY_AUTH_INCLUDE] is not set. Check env.local file.
WARNING:  Optional variable [THREDDS_ADDITIONAL_CATALOG] is not set. Check env.local file.
WARNING:  Optional variable [JUPYTER_LOGIN_BANNER_TOP_SECTION] is not set. Check env.local file.
WARNING:  Optional variable [SECURE_DATA_PROXY_AUTH_INCLUDE] is not set. Check env.local file.
WARNING:  Optional variable [AUTODEPLOY_EXTRA_REPOS_AS_DOCKER_VOLUMES] is not set. Check env.local file.
WARNING:  Optional variable [AUTODEPLOY_PLATFORM_EXTRA_DOCKER_ARGS] is not set. Check env.local file.
WARNING:  Optional variable [ALERTMANAGER_EXTRA_ROUTES] is not set. Check env.local file.
WARNING:  Optional variable [ALERTMANAGER_EXTRA_INHIBITION] is not set. Check env.local file.
WARNING:  Optional variable [ALERTMANAGER_EXTRA_RECEIVERS] is not set. Check env.local file.

mishaschwartz commented 9 months ago

@fmigneault Thank you for all your work on this! @tlvu Thanks for your reviews!

@fmigneault I'm still happy with this so feel free to merge whenever you're able

mishaschwartz commented 9 months ago

I don't think I should be doing this test for you. We have a fully reproducible stack so that we can be independent in our testing.

@tlvu I understand where your coming from. I think the difference here is that all the other tests we rely on are part of an automated CI pipeline and the autodeploy tests are the only tests that have to be run manually. If we could make the autodeploy tests automated then we wouldn't have the issue of having to ask any one in particular to verify that test before each PR.

tlvu commented 9 months ago

@tlvu I understand where your coming from. I think the difference here is that all the other tests we rely on are part of an automated CI pipeline and the autodeploy tests are the only tests that have to be run manually. If we could make the autodeploy tests automated then we wouldn't have the issue of having to ask any one in particular to verify that test before each PR.

Thanks @mishaschwartz, you've touched on an important point here. Allow me to elaborate further.

To put it more clearly, we only think about fresh new deployment scenario and we forget about upgrade scenario of existing system already in production. Autodeploy is just one point of this upgrade scenario. Example of some other points related to upgrade: DB upgrade that can handle existing data, backward compatible for "public API" (ex: the location of read-configs.include.sh, avoid renaming any variables that is in default.env, renaming of components should leave old paths in .gitignore, ...) and so on.

The fact that Cowbird is not backward compatible with existing Jupyter users (https://github.com/bird-house/birdhouse-deploy/issues/425) is another example of us forgetting about the upgrade scenario.

So I think we should always look further than just getting the automated CI pipeline to pass because that CI pipeline do not include the upgrade scenario.

fmigneault commented 9 months ago

@tlvu I am happy to test and fix anything that is handled by the CI, however auto-deploy is an entire other feature that is not used on our end. I cannot start investing sufficient time to handle a deployment strategy that essentially replicates what the CI does already to spawn the instance. Also, like mentioned, the upgrade scenario requires an active instance. We do not have such requirement (even our hirondelle instance is considered a development node that can go down any time for other work unrelated to DACCS, with its own deployment method), therefore I have no instance to run auto-deploy against.

If the CI did a small test to run a deployment (even a pseudo-one if needed) and report errors, then I could plan fixes for it, but I currently have no automated way to get this feedback. I am not expecting anyone to run end-to-end tests of the full stack or individual components that are not handled in some automated way, because that takes too much time and effort, as it is not reproducible anyway.

Since auto-deploy seems that critical on Ouranos' side, I believe it would be worth for you to invest time in developing a test case that does the necessary checks to ensure it works, because even a custom setup on my end would not guarantee it works on yours (e.g.: curl would have been available for me taking the same weaver post script example). In the end, you know best which items to verify on your end to consider auto-deploy working as intended. Note that I would be happy to add more tests to such a CI test suite if some deploy considerations were raised and if that test suite was defined, but budget-wise, I cannot invest the time required to develop the initial definition.