move STAC service Magpie definition under its component

[fmigneault]

Overview

Move STAC service definitions under components/stac configuration.

Changes

Non-breaking changes

• Move initial stac service Magpie definition under its component configuration.

  • Before this change, optional-components/stac-public-access was mandatory since the stac service under Magpie was not created otherwise, leading to "service not found" error when requesting the /stac endpoint.
  • Ensure that the first stac resource under stac service in Magpie is created by default. Without this resource being defined initially, it is very easy to forget creating it, which would not take into account the required /stac/stac request path to properly resolve the real endpoints where STAC API is served.

• Remove optional-components/stac-public-access dependency under optional-components/all-public-access to avoid indirectly enforcing components/stac when optional-components/all-public-access is enabled. Users that desire using optional-components/stac-public-access will have to add it explicitly to the list of EXTRA_CONF_DIRS.

• Rename optional-components/stac-public-access/config/magpie/config.yml.template to optional-components/stac-public-access/config/magpie/permissions.cfg in order to align with permissions-specific contents as accomplished with other components.

Breaking changes

• n/a

Related Issue / Discussion

• Relates to #382

[fmigneault]

@mishaschwartz I believe stac-public-access should not be removed. Instead, its dependency in all-public-access should be removed. This is to avoid stac being enforced by all-public-access, while still providing a quick method to make it public as needed.

[crim-jenkins-bot]

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2076/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : stac-magpie-provider-without-public-permissions
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-46.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1319/

NOTEBOOK TEST RESULTS

    
[2023-09-20T15:41:34.472Z] ============================= test session starts ==============================
[2023-09-20T15:41:34.472Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-09-20T15:41:34.472Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master@2
[2023-09-20T15:41:34.472Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-09-20T15:41:34.472Z] collected 237 items
[2023-09-20T15:41:34.472Z] 
[2023-09-20T15:41:44.412Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2023-09-20T15:41:52.603Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2023-09-20T15:42:02.320Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 10%]
[2023-09-20T15:42:10.427Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .F......       [ 13%]
[2023-09-20T16:15:26.286Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 18%]
[2023-09-20T16:15:28.827Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 18%]
[2023-09-20T16:15:56.036Z] ...............                                                          [ 25%]
[2023-09-20T16:16:09.338Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 27%]
[2023-09-20T16:16:16.059Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 29%]
[2023-09-20T16:16:31.301Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 32%]
[2023-09-20T16:16:32.682Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 32%]
[2023-09-20T16:16:38.909Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 35%]
[2023-09-20T16:16:43.583Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 37%]
[2023-09-20T16:20:15.103Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 44%]
[2023-09-20T16:21:30.512Z] .............                                                            [ 49%]
[2023-09-20T16:21:33.254Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 51%]
[2023-09-20T16:21:35.574Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 54%]
[2023-09-20T16:21:52.725Z] .................                                                        [ 62%]
[2023-09-20T16:22:00.562Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 64%]
[2023-09-20T16:22:01.944Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 64%]
[2023-09-20T16:22:03.293Z] .FFFFFFFF                                                                [ 68%]
[2023-09-20T16:22:13.432Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 73%]
[2023-09-20T16:22:22.764Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 75%]
[2023-09-20T16:22:24.183Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 76%]
[2023-09-20T16:22:27.241Z] ......                                                                   [ 78%]
[2023-09-20T16:22:33.828Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 79%]
[2023-09-20T16:22:48.836Z] .............                                                            [ 84%]
[2023-09-20T16:22:58.832Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 85%]
[2023-09-20T16:23:33.556Z] ....s.                                                                   [ 87%]
[2023-09-20T16:23:41.688Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2023-09-20T16:23:56.453Z] ...                                                                      [ 89%]
[2023-09-20T16:24:08.682Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 89%]
[2023-09-20T16:24:31.013Z] ......                                                                   [ 92%]
[2023-09-20T16:24:33.859Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-09-20T16:27:13.452Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2023-09-20T16:27:13.452Z] 
[2023-09-20T16:27:13.452Z] =================================== FAILURES ===================================
    
  

[mishaschwartz]

Instead, its dependency in all-public-access should be removed.

Ok I'm fine with that compromise. Can you remove it from the default.conf file then?

[crim-jenkins-bot]

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2077/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : stac-magpie-provider-without-public-permissions
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-133.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1320/

NOTEBOOK TEST RESULTS

    
[2023-09-20T19:29:32.859Z] ============================= test session starts ==============================
[2023-09-20T19:29:32.859Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-09-20T19:29:32.859Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2023-09-20T19:29:32.859Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-09-20T19:29:32.859Z] collected 237 items
[2023-09-20T19:29:32.859Z] 
[2023-09-20T19:29:40.733Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2023-09-20T19:29:49.687Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2023-09-20T19:29:59.399Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 10%]
[2023-09-20T19:30:06.570Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .F......       [ 13%]
[2023-09-20T19:41:18.256Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 18%]
[2023-09-20T19:44:09.787Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 18%]
[2023-09-20T19:44:09.787Z] ...............                                                          [ 25%]
[2023-09-20T19:44:18.272Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 27%]
[2023-09-20T19:44:25.180Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 29%]
[2023-09-20T19:44:41.534Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 32%]
[2023-09-20T19:44:43.189Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 32%]
[2023-09-20T19:44:50.859Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 35%]
[2023-09-20T19:44:55.182Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 37%]
[2023-09-20T19:49:42.546Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 44%]
[2023-09-20T19:51:03.966Z] .............                                                            [ 49%]
[2023-09-20T19:51:07.786Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 51%]
[2023-09-20T19:51:10.379Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 54%]
[2023-09-20T19:51:26.963Z] .................                                                        [ 62%]
[2023-09-20T19:51:35.057Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 64%]
[2023-09-20T19:51:36.450Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 64%]
[2023-09-20T19:51:37.801Z] .FFFFFFFF                                                                [ 68%]
[2023-09-20T19:51:48.628Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 73%]
[2023-09-20T19:51:57.980Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 75%]
[2023-09-20T19:51:59.360Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 76%]
[2023-09-20T19:52:02.660Z] ......                                                                   [ 78%]
[2023-09-20T19:52:09.232Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 79%]
[2023-09-20T19:52:25.020Z] .............                                                            [ 84%]
[2023-09-20T19:52:37.248Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 85%]
[2023-09-20T19:53:23.378Z] ....s.                                                                   [ 87%]
[2023-09-20T19:53:31.644Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2023-09-20T19:53:47.380Z] ...                                                                      [ 89%]
[2023-09-20T19:54:02.281Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 89%]
[2023-09-20T19:54:26.232Z] ......                                                                   [ 92%]
[2023-09-20T19:54:29.493Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-09-20T19:57:19.182Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2023-09-20T19:57:19.182Z] 
[2023-09-20T19:57:19.182Z] =================================== FAILURES ===================================
    
  

[crim-jenkins-bot]

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2080/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : stac-magpie-provider-without-public-permissions
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-46.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1322/

NOTEBOOK TEST RESULTS

    
[2023-09-20T20:19:51.789Z] ============================= test session starts ==============================
[2023-09-20T20:19:51.789Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-09-20T20:19:51.789Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2023-09-20T20:19:51.789Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-09-20T20:19:51.789Z] collected 237 items
[2023-09-20T20:19:51.789Z] 
[2023-09-20T20:19:59.861Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2023-09-20T20:20:11.894Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2023-09-20T20:20:24.144Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 10%]
[2023-09-20T20:20:30.726Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .F......       [ 13%]
[2023-09-20T20:33:44.485Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 18%]
[2023-09-20T20:33:44.485Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 18%]
[2023-09-20T20:36:23.234Z] ...............                                                          [ 25%]
[2023-09-20T20:36:33.054Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 27%]
[2023-09-20T20:36:40.325Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 29%]
[2023-09-20T20:36:56.300Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 32%]
[2023-09-20T20:36:57.680Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 32%]
[2023-09-20T20:37:04.228Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 35%]
[2023-09-20T20:37:08.473Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 37%]
[2023-09-20T20:39:55.436Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 44%]
[2023-09-20T20:41:17.770Z] .............                                                            [ 49%]
[2023-09-20T20:41:21.856Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 51%]
[2023-09-20T20:41:24.078Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 54%]
[2023-09-20T20:41:39.902Z] .................                                                        [ 62%]
[2023-09-20T20:41:47.483Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 64%]
[2023-09-20T20:41:48.863Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 64%]
[2023-09-20T20:42:03.545Z] .........                                                                [ 68%]
[2023-09-20T20:42:12.861Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 73%]
[2023-09-20T20:42:21.828Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 75%]
[2023-09-20T20:42:23.207Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 76%]
[2023-09-20T20:42:26.859Z] ......                                                                   [ 78%]
[2023-09-20T20:42:34.998Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 79%]
[2023-09-20T20:42:49.463Z] .............                                                            [ 84%]
[2023-09-20T20:43:01.679Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 85%]
[2023-09-20T20:43:38.306Z] ....s.                                                                   [ 87%]
[2023-09-20T20:43:46.437Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2023-09-20T20:43:59.735Z] ...                                                                      [ 89%]
[2023-09-20T20:44:14.639Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 89%]
[2023-09-20T20:44:37.652Z] ......                                                                   [ 92%]
[2023-09-20T20:44:40.402Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-09-20T20:47:14.507Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2023-09-20T20:47:14.507Z] 
[2023-09-20T20:47:14.507Z] =================================== FAILURES ===================================