search

opendatacube / datacube-ows

Open Data Cube Open Web Services
Other
70 stars 37 forks source link

lack of documentation on `root_layer_folder` #487

Closed pindge closed 3 years ago

pindge commented 3 years ago

datacube-ows version: latest ows_config.py version/link:: splitting config to multi files

Description

AttributeError
AttributeError: 'OWSConfig' object has no attribute 'root_layer_folder'

image

ows_cfg.py

 {
                    "title": "Collection 3",
                    "abstract": """

                    """,
                    "layers": [
                          {
                        "include": "c3_cfg.dea_c3_ls5_ard",
                        "type": "python",
                       },

                       {
                        "include": "c3_cfg.dea_c3_ls7_ard",
                        "type": "python",
                       },
                       {
                        "include": "c3_cfg.dea_c3_ls8_ard",
                        "type": "python",
                       },

#                         {
#                             "title": "DEA C3 Landsat 8 ARD",
#                             "abstract": """
# This product takes Landsat 8 imagery captured over the Australian continent and corrects for inconsistencies across land and coastal fringes. The result is accurate and standardised surface reflectance data, which is instrumental in identifying and quantifying environmental change.

# The imagery is captured using the Operational Land Imager (OLI) and Thermal Infra-Red Scanner (TIRS) sensors aboard Landsat 8.

# This product is a single, cohesive Analysis Ready Data (ARD) package, which allows you to analyse surface reflectance data as is, without the need to apply additional corrections.

# It contains three sub-products that provide corrections or attribution information:

# Surface Reflectance NBAR 3 (Landsat 8 OLI-TIRS)
# Surface Reflectance NBART 3 (Landsat 8 OLI-TIRS)
# Surface Reflectance OA 3 (Landsat 8 OLI-TIRS)
# The resolution is a 30 m grid based on the USGS Landsat Collection 1 archive.""",
#                             # The WMS name for the layer
#                             "name": "ga_ls8c_ard_3",
#                             # The Datacube name for the associated data product
#                             "product_name": "ga_ls8c_ard_3",
#                             "bands": bands_c3_ls_8,
#                             "resource_limits": reslim_tmad,
#                             "image_processing": {
#                                 "extent_mask_func": "datacube_ows.ogc_utils.mask_by_val",
#                                 "always_fetch_bands": [],
#                                 "manual_merge": False,
#                             },
#                             "wcs": {
#                                 "native_crs": "EPSG:3577",
#                                 "native_resolution": [25, -25],
#                                 "default_bands": ["nbart_red", "nbart_green", "nbart_blue"],
#                             },
#                             "styling": {
#                                 "default_style": "simple_rgb",
#                                 "styles": style_c3_ls_8
#                             },
#                         },
#                         {
#                             "title": "DEA C3 Landsat 7 ARD",
#                             "abstract": """
# The United States Geological Survey's (USGS) Landsat satellite program has been capturing images of the Australian continent for more than 30 years. This data is highly useful for land and coastal mapping studies.
# In particular, the light reflected from the Earth’s surface (surface reflectance) is important for monitoring environmental resources – such as agricultural production and mining activities – over time.

# We need to make accurate comparisons of imagery acquired at different times, seasons and geographic locations. However, inconsistencies can arise due to variations in atmospheric conditions, sun position, sensor view angle, surface slope and surface aspect. These need to be reduced or removed to ensure the data is consistent and can be compared over time.

# For service status information, see https://status.dea.ga.gov.au""",
#                             # The WMS name for the layer
#                             "name": "ga_ls7e_ard_3",
#                             # The Datacube name for the associated data product
#                             "product_name": "ga_ls7e_ard_3",
#                             "bands": bands_c3_ls_7,
#                             "resource_limits": reslim_tmad,
#                             "image_processing": {
#                                 "extent_mask_func": "datacube_ows.ogc_utils.mask_by_val",
#                                 "always_fetch_bands": [],
#                                 "manual_merge": False,
#                             },
#                             "wcs": {
#                                 "native_crs": "EPSG:3577",
#                                 "native_resolution": [25, -25],
#                                 "default_bands": ["nbart_red", "nbart_green", "nbart_blue"],
#                             },
#                             "styling": {
#                                 "default_style": "simple_rgb",
#                                 "styles": style_c3_ls_7
#                             },
#                         },
#                         {
#                             "title": "DEA C3 Landsat 5 ARD",
#                             "abstract": """
# The United States Geological Survey's (USGS) Landsat satellite program has been capturing images of the Australian continent for more than 30 years. This data is highly useful for land and coastal mapping studies.

# In particular, the light reflected from the Earth’s surface (surface reflectance) is important for monitoring environmental resources – such as agricultural production and mining activities – over time.

# We need to make accurate comparisons of imagery acquired at different times, seasons and geographic locations. However, inconsistencies can arise due to variations in atmospheric conditions, sun position, sensor view angle, surface slope and surface aspect. These need to be reduced or removed to ensure the data is consistent and can be compared over time.

# For service status information, see https://status.dea.ga.gov.au""",
#                             # The WMS name for the layer
#                             "name": "ga_ls5t_ard_3",
#                             # The Datacube name for the associated data product
#                             "product_name": "ga_ls5t_ard_3",
#                             "bands": bands_c3_ls_common,
#                             "resource_limits": reslim_tmad,
#                             "image_processing": {
#                                 "extent_mask_func": "datacube_ows.ogc_utils.mask_by_val",
#                                 "always_fetch_bands": [],
#                                 "manual_merge": False,
#                             },
#                             "wcs": {
#                                 "native_crs": "EPSG:3577",
#                                 "native_resolution": [25, -25],
#                                 "default_bands": ["nbart_red", "nbart_green", "nbart_blue"],
#                             },
#                             "styling": {
#                                 "default_style": "simple_rgb",
#                                 "styles": style_c3_ls_common
#                             },
#                         },
                    ]

c3_cfg.py

from legend_cfg import legend_idx_0_1_5ticks

bands_c3_ls_common = {
    "nbart_blue": ["nbart_blue"],
    "nbart_green": ["nbart_green"],
    "nbart_red": ["nbart_red"],
    "nbart_nir": ["nbart_nir", "nbart_near_infrared"],
    "nbart_swir_1": ["nbart_swir_1", "nbart_shortwave_infrared_1"],
    "nbart_swir_2": ["nbart_swir_2", "nbart_shortwave_infrared_2"],
}

bands_c3_ls_7 = bands_c3_ls_common.copy()
bands_c3_ls_7.update({
    "nbart_panchromatic": [],
})

bands_c3_ls_8 = bands_c3_ls_7.copy()
bands_c3_ls_8.update({
    "nbart_coastal_aerosol": ["coastal_aerosol",  "nbart_coastal_aerosol"],
})

style_c3_ls_common = [
    style_c3_simple_rgb,
    style_c3_false_colour,
    style_c3_ndvi,
    style_c3_ndwi,
    style_c3_mndwi,
    style_c3_pure_blue,
    style_c3_pure_green,
    style_c3_pure_red,
    style_c3_pure_nir,
    style_c3_pure_swir1,
    style_c3_pure_swir2,
]

style_c3_ls_7 = style_c3_ls_common.append(style_c3_pure_panchromatic)
# style_c3_ls_8 = style_c3_ls_7.append(style_c3_pure_aerosol)
style_c3_ls_8 = [].append(style_c3_pure_aerosol)

reslim_c3_ls = {
    "wms": {
        "zoomed_out_fill_colour": [150, 180, 200, 160],
        "min_zoom_factor": 15.0,
        # "max_datasets": 16, # Defaults to no dataset limit
    },
    "wcs": {
        # "max_datasets": 16, # Defaults to no dataset limit
    },
}

dea_c3_ls8_ard = {
                            "title": "DEA C3 Landsat 8 ARD",
                            "abstract": """
This product takes Landsat 8 imagery captured over the Australian continent and corrects for inconsistencies across land and coastal fringes. The result is accurate and standardised surface reflectance data, which is instrumental in identifying and quantifying environmental change.

The imagery is captured using the Operational Land Imager (OLI) and Thermal Infra-Red Scanner (TIRS) sensors aboard Landsat 8.

This product is a single, cohesive Analysis Ready Data (ARD) package, which allows you to analyse surface reflectance data as is, without the need to apply additional corrections.

It contains three sub-products that provide corrections or attribution information:

Surface Reflectance NBAR 3 (Landsat 8 OLI-TIRS)
Surface Reflectance NBART 3 (Landsat 8 OLI-TIRS)
Surface Reflectance OA 3 (Landsat 8 OLI-TIRS)
The resolution is a 30 m grid based on the USGS Landsat Collection 1 archive.""",
                            # The WMS name for the layer
                            "name": "ga_ls8c_ard_3",
                            # The Datacube name for the associated data product
                            "product_name": "ga_ls8c_ard_3",
                            "bands": bands_c3_ls_8,
                            "resource_limits": reslim_c3_ls,
                            "image_processing": {
                                "extent_mask_func": "datacube_ows.ogc_utils.mask_by_val",
                                "always_fetch_bands": [],
                                "manual_merge": False,
                            },
                            "wcs": {
                                "native_crs": "EPSG:3577",
                                "native_resolution": [25, -25],
                                "default_bands": ["nbart_red", "nbart_green", "nbart_blue"],
                            },
                            "styling": {
                                "default_style": "simple_rgb",
                                "styles": style_c3_ls_8
                            },
                        },
dea_c3_ls7_ard =        {
                            "title": "DEA C3 Landsat 7 ARD",
                            "abstract": """
The United States Geological Survey's (USGS) Landsat satellite program has been capturing images of the Australian continent for more than 30 years. This data is highly useful for land and coastal mapping studies.
In particular, the light reflected from the Earth’s surface (surface reflectance) is important for monitoring environmental resources – such as agricultural production and mining activities – over time.

We need to make accurate comparisons of imagery acquired at different times, seasons and geographic locations. However, inconsistencies can arise due to variations in atmospheric conditions, sun position, sensor view angle, surface slope and surface aspect. These need to be reduced or removed to ensure the data is consistent and can be compared over time.

For service status information, see https://status.dea.ga.gov.au""",
                            # The WMS name for the layer
                            "name": "ga_ls7e_ard_3",
                            # The Datacube name for the associated data product
                            "product_name": "ga_ls7e_ard_3",
                            "bands": bands_c3_ls_7,
                            "resource_limits": reslim_c3_ls,
                            "image_processing": {
                                "extent_mask_func": "datacube_ows.ogc_utils.mask_by_val",
                                "always_fetch_bands": [],
                                "manual_merge": False,
                            },
                            "wcs": {
                                "native_crs": "EPSG:3577",
                                "native_resolution": [25, -25],
                                "default_bands": ["nbart_red", "nbart_green", "nbart_blue"],
                            },
                            "styling": {
                                "default_style": "simple_rgb",
                                "styles": style_c3_ls_7
                            },
                        },
dea_c3_ls5_ard =        {
                            "title": "DEA C3 Landsat 5 ARD",
                            "abstract": """
The United States Geological Survey's (USGS) Landsat satellite program has been capturing images of the Australian continent for more than 30 years. This data is highly useful for land and coastal mapping studies.

In particular, the light reflected from the Earth’s surface (surface reflectance) is important for monitoring environmental resources – such as agricultural production and mining activities – over time.

We need to make accurate comparisons of imagery acquired at different times, seasons and geographic locations. However, inconsistencies can arise due to variations in atmospheric conditions, sun position, sensor view angle, surface slope and surface aspect. These need to be reduced or removed to ensure the data is consistent and can be compared over time.

For service status information, see https://status.dea.ga.gov.au""",
                            # The WMS name for the layer
                            "name": "ga_ls5t_ard_3",
                            # The Datacube name for the associated data product
                            "product_name": "ga_ls5t_ard_3",
                            "bands": bands_c3_ls_common,
                            "resource_limits": reslim_c3_ls,
                            "image_processing": {
                                "extent_mask_func": "datacube_ows.ogc_utils.mask_by_val",
                                "always_fetch_bands": [],
                                "manual_merge": False,
                            },
                            "wcs": {
                                "native_crs": "EPSG:3577",
                                "native_resolution": [25, -25],
                                "default_bands": ["nbart_red", "nbart_green", "nbart_blue"],
                            },
                            "styling": {
                                "default_style": "simple_rgb",
                                "styles": style_c3_ls_common
                            },
                        },

What I Did

searching for the word root_layer_folder only returned https://github.com/opendatacube/datacube-ows/search?q=root_layer_folder, no information on how to use it


993 | def make_ready(self, dc, *args, **kwargs):
-- | --
994 | self.native_product_index = {}
995 | self.root_layer_folder.make_ready(dc, *args, **kwargs)
… |  
1114 | self.declare_unready("native_product_index")
1115 | self.root_layer_folder = OWSFolder({
1116 | "title": "Root Folder (hidden)"
SpacemanPaul commented 3 years ago

The problem appears to be trailing commas turning dictionaries into lists, which is not checked by the config parser.

It's a bit painful to check for, but probably worth it considering how easy a mistake it is to make - a more helpful error message that doesn't reference OWS internals would be good.

SpacemanPaul commented 3 years ago

Fixed in #692