search

bird-house / birdhouse-deploy

Scripts and configurations to deploy the various birds and servers required for a full-fledged production platform
https://birdhouse-deploy.readthedocs.io/en/latest/
Apache License 2.0
4 stars 7 forks source link

Update Thredds to supported version #413

Open mishaschwartz opened 11 months ago

mishaschwartz commented 11 months ago

Overview

Unidata has dropped support for TDS versions < 5.x. This updates Thredds to version 5.4.

Changes

Non-breaking changes

Breaking changes

Related Issue / Discussion

Additional Information

Links to other issues or sources.

birdhouse_daccs_configs_branch: master birdhouse_skip_ci: false

tlvu commented 11 months ago

FYI, for this change, the automated pipeline test is not enough. .ncml tests are not done against the new Thredds in this PR but against the production server with the current Thredds to avoid having to deploy all the big data required by .ncml on test servers.

We have already attempted to upgrade Thredds on Ouranos side and we have .ncml issues and, if my memory is right, there are also 2 notebooks not using .ncml failing as well which should be caught by the pipeline.

mishaschwartz commented 11 months ago

@tlvu

Thanks for the info. I mostly created this PR to test the pipeline with the updated version to see what happens. I can close this if you're working on it on the ouranos side.

tlvu commented 11 months ago

Oh keep this PR, we have not yet opened a formal PR on our side, we simply override in env.local for testing. We can piggyback on your PR once we sort out the various issues on our side.

crim-jenkins-bot commented 11 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2366/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-thredds-5.4
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-20.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1482/

NOTEBOOK TEST RESULTS 
    
[2023-12-14T20:21:16.415Z] ============================= test session starts ==============================
[2023-12-14T20:21:16.415Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-12-14T20:21:16.415Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2023-12-14T20:21:16.415Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-12-14T20:21:16.415Z] collected 265 items
[2023-12-14T20:21:16.415Z] 
[2023-12-14T20:21:27.112Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2023-12-14T20:21:54.191Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2023-12-14T20:22:02.837Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2023-12-14T20:22:11.544Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2023-12-14T20:22:21.246Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2023-12-14T20:22:29.940Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2023-12-14T20:29:53.778Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2023-12-14T20:29:53.778Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2023-12-14T20:29:59.420Z] ...............                                                          [ 33%]
[2023-12-14T20:30:09.837Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2023-12-14T20:30:17.178Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2023-12-14T20:30:33.269Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2023-12-14T20:30:34.675Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 40%]
[2023-12-14T20:30:39.719Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2023-12-14T20:30:44.519Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2023-12-14T20:34:02.266Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2023-12-14T20:35:18.259Z] .............                                                            [ 55%]
[2023-12-14T20:35:20.033Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb F.FF             [ 56%]
[2023-12-14T20:35:22.616Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2023-12-14T20:35:39.762Z] .................                                                        [ 66%]
[2023-12-14T20:35:46.780Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2023-12-14T20:35:48.173Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2023-12-14T20:36:05.962Z] .........                                                                [ 72%]
[2023-12-14T20:36:15.808Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2023-12-14T20:36:25.188Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2023-12-14T20:36:27.107Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2023-12-14T20:36:30.198Z] ......                                                                   [ 81%]
[2023-12-14T20:36:38.783Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2023-12-14T20:36:52.964Z] .............                                                            [ 86%]
[2023-12-14T20:37:02.969Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2023-12-14T20:37:38.282Z] ....s.                                                                   [ 89%]
[2023-12-14T20:37:46.429Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2023-12-14T20:38:00.039Z] ...                                                                      [ 90%]
[2023-12-14T20:38:14.968Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2023-12-14T20:38:37.896Z] ......                                                                   [ 93%]
[2023-12-14T20:38:39.512Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-12-14T20:41:28.996Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2023-12-14T20:41:28.996Z] 
[2023-12-14T20:41:28.996Z] =================================== FAILURES ===================================
tlvu commented 10 months ago

A few problems found related to Thredds 5.4 https://github.com/Unidata/tds/issues/449 https://github.com/Unidata/tds/issues/406

Notebook changes required: https://github.com/Ouranosinc/pavics-sdi/pull/317 https://github.com/Ouranosinc/PAVICS-landing/pull/77

crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2420/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-thredds-5.4
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-46.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1498/

NOTEBOOK TEST RESULTS 
    
[2024-01-17T22:33:10.547Z] ============================= test session starts ==============================
[2024-01-17T22:33:10.547Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-17T22:33:10.547Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-01-17T22:33:10.547Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-17T22:33:10.547Z] collected 264 items
[2024-01-17T22:33:10.547Z] 
[2024-01-17T22:33:20.928Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-17T22:33:54.638Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-17T22:33:59.781Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-17T22:34:09.377Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-17T22:34:19.740Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-17T22:34:26.438Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-01-17T22:47:35.169Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-17T22:47:35.432Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-17T22:47:44.008Z] ...............                                                          [ 33%]
[2024-01-17T22:47:54.172Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-17T22:48:01.464Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-17T22:48:18.849Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-17T22:48:24.608Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-17T22:48:29.156Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-17T22:52:13.417Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-17T22:53:30.247Z] .............                                                            [ 54%]
[2024-01-17T22:53:35.021Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ..FF             [ 56%]
[2024-01-17T22:53:37.608Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-17T22:53:54.973Z] .................                                                        [ 65%]
[2024-01-17T22:54:03.287Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-17T22:54:04.670Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-17T22:54:16.606Z] ........F                                                                [ 71%]
[2024-01-17T22:54:27.479Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-17T22:54:37.108Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-17T22:54:39.038Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-17T22:54:42.106Z] ......                                                                   [ 81%]
[2024-01-17T22:54:50.251Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-17T22:55:06.258Z] .............                                                            [ 86%]
[2024-01-17T22:55:18.512Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-17T22:56:04.236Z] ....s.                                                                   [ 89%]
[2024-01-17T22:56:12.412Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-17T22:56:27.944Z] ...                                                                      [ 90%]
[2024-01-17T22:56:42.852Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-17T22:57:07.610Z] ......                                                                   [ 93%]
[2024-01-17T22:57:10.373Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-17T22:59:59.854Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-17T22:59:59.854Z] 
[2024-01-17T22:59:59.854Z] =================================== FAILURES ===================================
crim-jenkins-bot commented 10 months ago

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2421/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-thredds-5.4
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-69.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1499/

NOTEBOOK TEST RESULTS 
    
[2024-01-17T22:34:14.373Z] ============================= test session starts ==============================
[2024-01-17T22:34:14.373Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-01-17T22:34:14.373Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master@2
[2024-01-17T22:34:14.373Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-01-17T22:34:14.373Z] collected 264 items
[2024-01-17T22:34:14.373Z] 
[2024-01-17T22:34:26.345Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-01-17T22:34:56.733Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-01-17T22:35:02.166Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-01-17T22:35:16.066Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 17%]
[2024-01-17T22:35:30.408Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 19%]
[2024-01-17T22:35:40.989Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 22%]
[2024-01-17T22:47:57.270Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 27%]
[2024-01-17T22:47:57.270Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 27%]
[2024-01-17T22:48:04.968Z] ...............                                                          [ 33%]
[2024-01-17T22:48:15.219Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 35%]
[2024-01-17T22:48:21.966Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 37%]
[2024-01-17T22:48:39.144Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 39%]
[2024-01-17T22:48:43.930Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 42%]
[2024-01-17T22:48:48.742Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 44%]
[2024-01-17T22:52:08.226Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 50%]
[2024-01-17T22:53:27.783Z] .............                                                            [ 54%]
[2024-01-17T22:53:33.197Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ..FF             [ 56%]
[2024-01-17T22:53:36.317Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 59%]
[2024-01-17T22:53:53.937Z] .................                                                        [ 65%]
[2024-01-17T22:54:02.283Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 68%]
[2024-01-17T22:54:03.691Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 68%]
[2024-01-17T22:54:22.062Z] ........F                                                                [ 71%]
[2024-01-17T22:54:31.540Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-01-17T22:54:41.373Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-01-17T22:54:42.752Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-01-17T22:54:46.046Z] ......                                                                   [ 81%]
[2024-01-17T22:54:52.844Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-01-17T22:55:09.289Z] .............                                                            [ 86%]
[2024-01-17T22:55:21.529Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-01-17T22:56:06.782Z] ....s.                                                                   [ 89%]
[2024-01-17T22:56:16.828Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-01-17T22:56:31.615Z] ...                                                                      [ 90%]
[2024-01-17T22:56:46.528Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-01-17T22:57:11.326Z] ......                                                                   [ 93%]
[2024-01-17T22:57:14.072Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-01-17T23:00:00.239Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-01-17T23:00:00.239Z] 
[2024-01-17T23:00:00.239Z] =================================== FAILURES ===================================
tlvu commented 8 months ago

FYI @mishaschwartz we still have not done fixing the issues on our side. Besides there is a performce problem with 5.4 (https://github.com/Unidata/tds/issues/406) and we hope to be fixed in 5.5 so this PR is not ready (5.5 is not released).

tlvu commented 6 days ago

FYI, we found more problems with version 5. NCML, UDDC, ISO and NetcdfSubset link do not work with v5. Have not had time to investigate. Opendap and other links work fine.

Ex: click on the NCML, UDDC, ISO and NetcdfSubset link here https://pavics.ouranos.ca/testthredds/catalog/testdatasets/testdata/catalog.html?dataset=testdatasets/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc

tlvu commented 5 days ago

FYI, we found more problems with version 5. NCML, UDDC, ISO and NetcdfSubset link do not work with v5. Have not had time to investigate. Opendap and other links work fine.

Ex: click on the NCML, UDDC, ISO and NetcdfSubset link here https://pavics.ouranos.ca/testthredds/catalog/testdatasets/testdata/catalog.html?dataset=testdatasets/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc

Related to https://github.com/Unidata/thredds-docker/issues/310

tlvu commented 5 days ago

NCML, UDDC, ISO links fixed by adding the missing jar, see https://github.com/Unidata/thredds-docker/issues/310#issuecomment-2486367398

No other clues found for broken NetcdfSubset link, opened an issue https://github.com/Unidata/tds/issues/544