Overhaul layer description configuration

[minniewong]

1. Automatically pull temporal range from GetCapabilities
2. Split up layer descriptions by layer
3. Move metadata description tag to layer file
4. List category that the measurement fits in, in the measurement file
5. etc...

[minniewong]

327

[ZachTRice]

Current structure of the metadata, layers, and measurements.

Metadata files: config/metadata/aster/GDEM.md (with references to 3 separate settings)

Layer Files: config/wv.json/layers/aster/ASTER_GDEM_Color_index.json config/wv.json/layers/aster/ASTER_GDEM_Color_Shaded_relief.json config/wv.json/layers/aster/ASTER_GDEM_Greyscale_Shaded_Relief.json

Layer file structure:
{
    "layers": {
        "ASTER_GDEM_Color_Index": {
            "id":       "ASTER_GDEM_Color_Index",
            "title":    "Global Digital Elevation Map (Color Index)",
            "subtitle": "Terra / ASTER",
            "tags":     "gdem",
            "group":    "overlays",
            "type":     "wmts",
            "product":  "ASTGTM",
            "wrapadjacentdays": true
        }
    }
}

Measurement file: config/wv.json/measurements/Golbal Digital Elevation Map.json

Measurement file structure:
{
    "measurements": {
        "Global Digital Elevation Map": {
            "id": "global-digital-elevation-map",
            "title": "Global Digital Elevation Map",
            "subtitle": "Terra / ASTER, Space Shuttle Endeavour / STS-99",
            "sources": {
                "Terra/ASTER": {
                    "id": "terra-aster",
                    "title": "Terra / ASTER",
                    "description": "aster/GDEM",
                    "image": "",
                    "settings": [
                        "ASTER_GDEM_Color_Index",
                        "ASTER_GDEM_Color_Shaded_Relief",
                        "ASTER_GDEM_Greyscale_Shaded_Relief"
                    ]
                },
                "Space Shuttle Endeavour/STS-99": {
                    "id": "space-shuttle-endeavour-sts-99",
                    "title": "Space Shuttle Endeavour / STS-99",
                    "description": "other/SRTM",
                    "image": "",
                    "settings": [
                        "SRTM_Color_Index"
                    ]
                }
            }
        }
    }
}

The biggest issue for me is that metadata is only referenced within the measurements .json file. Ideally, the 1 metadata md file would be broken out into 3 md files and then referenced within each individual layer.

We could keep a md where they are right now and leave the reference in the measurement .json file to it and use this as a source (measurement sub-category) description AND then break it down further into separate md files for each source. This structure would look like this...:

---

Structure of data with individual setting metadata files:

Metadata files: config/metadata/aster/GDEM.md (with references to 3 separate settings) config/metadata/aster/GDEM/ASTER_GDEM_Color_index.md NEW FILE HERE config/metadata/aster/GDEM/ASTER_GDEM_Color_Shaded_relief.md NEW FILE HERE config/metadata/aster/GDEM/ASTER_GDEM_Greyscale_Shaded_Relief.md *** NEW FILE HERE

Layer Files: config/wv.json/layers/aster/ASTER_GDEM_Color_index.json config/wv.json/layers/aster/ASTER_GDEM_Color_Shaded_relief.json config/wv.json/layers/aster/ASTER_GDEM_Greyscale_Shaded_Relief.json

Layer file structure:
{
    "layers": {
        "ASTER_GDEM_Color_Index": {
            "id":       "ASTER_GDEM_Color_Index",
            "title":    "Global Digital Elevation Map (Color Index)",
            "subtitle": "Terra / ASTER",
            "tags":     "gdem",
             "description": "aster/GDEM/ASTER_GDEM_Color_Index",  *** NEW METADATA REFERENCE HERE
            "group":    "overlays",
            "type":     "wmts",
            "product":  "ASTGTM",
            "wrapadjacentdays": true
        }
    }
}

Measurement file: config/wv.json/measurements/Golbal Digital Elevation Map.json

Measurement file structure:
{
    "measurements": {
        "Global Digital Elevation Map": {
            "id": "global-digital-elevation-map",
            "title": "Global Digital Elevation Map",
            "subtitle": "Terra / ASTER, Space Shuttle Endeavour / STS-99",
            "sources": {
                "Terra/ASTER": {
                    "id": "terra-aster",
                    "title": "Terra / ASTER",
                    "description": "aster/GDEM",
                    "image": "",
                    "settings": [
                        "ASTER_GDEM_Color_Index",
                        "ASTER_GDEM_Color_Shaded_Relief",
                        "ASTER_GDEM_Greyscale_Shaded_Relief"
                    ]
                },
                "Space Shuttle Endeavour/STS-99": {
                    "id": "space-shuttle-endeavour-sts-99",
                    "title": "Space Shuttle Endeavour / STS-99",
                    "description": "other/SRTM",
                    "image": "",
                    "settings": [
                        "SRTM_Color_Index"
                    ]
                }
            }
        }
    }
}

[ZachTRice]

We also need to decided whether measurements should have a description too. This would require adding a new top level, which would have to go into an entirely different directory since source meta descriptions are being cross referenced. See from the code block above:

"sources": {
        "Terra/ASTER": {
                "description": "aster/GDEM", ** ADDING A NEW TOP LEVEL HERE FOR MEASURMENTS WOULDN'T WORK
        },
        "Space Shuttle Endeavour/STS-99": {
                "description": "other/SRTM", ** ADDING A NEW TOP LEVEL HERE FOR MEASURMENTS WOULDN'T WORK
        },
}

[ZachTRice]

@tgunnoe - If you have some input on restructuring the data, we would definitely appreciate any input you have. We are going to have an in-person discussion about it and post our thoughts afterwards.

[Benjaki2]

Think about #384 as well

[ZachTRice]

Markdown files should not use an H1 tag. Multiple H1 tags on a single page is bad practice. Let's use an H3.

Ideally the logo would be wrapped in an H1 tag like so:

<h1>                   
  <a href="/" title="Worldview.earthdata.nasa.gov logo">
   <img src="logo.jpg" alt="Worldview logo" />
  </a>
</h1>

Then, set the H2 on the layer titles , with subtitles in a span. Then, within the descriptions we set an H3 tag. Look at this example:

[ZachTRice]

I think it would make sense to rename settings to layers within the measurements file.

[tgunnoe]

This would be my new layout

Layer:

{
    "layers": {
        "ASTER_GDEM_Color_Index": {
            "id":                     "ASTER_GDEM_Color_Index",
            "title":                  "Global Digital Elevation Map (Color Index)",
            "tags":                 "gdem",
             "metadata":        "aster/GDEM/ASTER_GDEM_Color_Index.md",  
            "group":               "overlays",
            "time"                  "day",
            "sources": [
                                       "terra_aster",
                                       "aster_gdem"
            ],
            "measurements": [
                                       "global_digital_elevation_map"
            ],
            "type":                "wmts",
            "reference":       "ASTGTM",
            "wrapdays":       "true"
        }
    }
}

Measurement:

{
    "measurements": {
        "global_digital_elevation_map": {
            "id":                   "global_digital_elevation_map",
            "title":                "Global Digital Elevation Map",
            "metadata":       "measurements/gdem.md",
            "categories": [
                                     "land_surface",
                                     "vegetation"
            ]
        }
    }
}

Sources (new):

{
    "sources": {
        "terra_aster": {
            "id":                "terra_aster",
            "title":             "Terra/ASTER",
            "metadata":    "sources/terra_aster.md",
            "provider":      "nasa",
            "status":         "active",
            "tracks":          {
                 "day": "terra_aster_day",
                 "night": "terra_aster_night"
            }
        }
    }
}

Category:

{
    "categories": {
        "air_quality": {
                "title":           "Air Quality",
                "id":              "air_quality",
                "image":       "images/air-quality.jpg",
                "metadata":  "categories/air_quality.md",
                "type":          "legacy"
        }
    }
}

[ZachTRice]

I made a diagram to visualize how the data is connected.

Here is the top-level layout of the config folder - description - palletes-custom.json - wv.json -- aoi -- categories -- layers -- measurements -- products -- projections -- sources -- tracks

I am assuming that tracks would be moved out of layers/reference/orbit and put into it's own tracks/ folder for better organization.

data_diagram.zip I have included the diagram PSD if anyone wants to download it and make changes.

[tgunnoe]

I think that since orbital tracks are still layers on the same par as openstreetmap layers and other reference layers, they dont need to be moved into their own object/folder. I was imagining that "tracks" under "source" objects would be a general reference to any other layers.

[ZachTRice]

Maybe the labeling of "tracks" should be changed to "references" since sources can refer to more than just orbital tracks? So then the tracks object I drew would go back to staying defined as a layers object?

It feels strange to me that sources has the reference back to tracks. I'm still a bit unclear on this pattern. Couldn't orbit tracks reference the source? And if layers need to reference tracks then add a track reference to the layer object too.

[ZachTRice]

Updated diagram to visualize the changes. Green = new Purple = changed Red = removed data_diagram_v2.zip

[tgunnoe]

I think it makes sense, but could be easy to argue that its not as consistent. However, I don't see why we'd consider Orbit Tracks the only unique reference layer.

the "source" as I see it can have worldview layers that is related to by default. So OpenStreetMap as a source, it would have references to Worldview layers such as Coastlines, Coastlines / Boarders / Roads, and Place names, for example. You could probably even argue that all reference layers in Worldview would work similarly: Here's a reference helper for this source of your data

@rboller @Benjaki2 @minniewong ?

[minniewong]

I went through a few different iterations using real layers to see how things would fit in the above schema.

Object: Layers

1. I think "reference" should be renamed back to "product"
2. We did call it "wrapadjacentdays" as opposed to "wrapdays" for a reason (I think because we wanted another option for wrapping static imagery #229, there may have been another reason that I don't remember)
3. Is "time" where we specify day/night so that the appropriate orbit track shows up? I think this needs a different name.
4. We need to keep the "inactive" tag in Layers as it is a per layer tag
5. WMS layers will have an additional set of tags like "projection". We need to check to see if we still need "transition", "startDate", "tileSize", "format", "palette". Object: Sources
6. I don't think we need description field, but then I can see it being useful only for the reference layers like coastlines, roads, orbit tracks.
7. Unsure of the necessity of "provider" and "status" here
8. There will probably very few instances where we'd have multiple sources for a layer, unless it's something like MODIS_Fires_All (which is combined Terra and Aqua). Object: Categories
9. Don't need "description" here
10. Rename the type "legacy" to "hazards" or something similar

[ZachTRice]

Ok, I will create a new PSD mockup with these changes.

Also I noticed during development that we have a sources object already, so the new sources object conflicts with this. This is the current sources object:

"sources": {
        "GIBS:arctic": {
            "url": "//gibs-{a-c}.earthdata.nasa.gov/wmts/epsg3413/best/wmts.cgi"
        },
        "GIBS:antarctic": {
            "url": "//gibs-{a-c}.earthdata.nasa.gov/wmts/epsg3031/best/wmts.cgi"
        },
        "GIBS:geographic": {
            "url": "//gibs-{a-c}.earthdata.nasa.gov/wmts/epsg4326/best/wmts.cgi"
        },
        "GIBS:wms": {
            "url": "//gibs-{a-c}.earthdata.nasa.gov/wms/wms.php"
        },
        "SEDAC:wms": {
            "url": "https://sedac.ciesin.columbia.edu/geoserver/wms"
        }
    }

During my testing I renamed this object "capabilities", does that naming make sense? If not, what else could we call this? @minniewong

[ZachTRice]

Updated diagram: data_diagram_v3.zip Blue color = changes we are discussing.

@minniewong Response to your list of suggestions (Great feedback!!): 👍 = changed. ❌ = not changed. ❓ = Questions (Lets all discuss). ℹ️ = Need more info.

Object: Layers

1. 👍 I think "reference" should be renamed back to "product"

2. 👍 We did call it "wrapadjacentdays" as opposed to "wrapdays" for a reason (I think because we wanted another option for wrapping static imagery #229, there may have been another reason that I don't remember)

3. ❓ Is "time" where we specify day/night so that the appropriate orbit track shows up? I think this needs a different name. -- I agree. Also tracks {day: [name], night: [name]} under sources should probably be named differently too.

4. 👍 We need to keep the "inactive" tag in Layers as it is a per layer tag -- (Was never removed just wasn't shown in diagram; updated diagram)

5. 👍 WMS layers will have an additional set of tags like "projection". We need to check to see if we still need "transition", "startDate", "tileSize", "format", "palette". -- (Was never removed just wasn't shown in diagram; updated diagram)

Object: Sources

6. ❓ I don't think we need description field, but then I can see it being useful only for the reference layers like coastlines, roads, orbit tracks. -- Should we have the source description reference a file even if it is not being used right now in case we want to use it later?

7. ℹ️ Unsure of the necessity of "provider" and "status" here -- Need more info from @tgunnoe on why this parameter was suggested.

Object: Categories

8. ❓ There will probably very few instances where we'd have multiple sources for a layer, unless it's something like MODIS_Fires_All (which is combined Terra and Aqua). -- Good to know. Probably best to keep it as a multi-format though for the future?

9. ❓ Don't need "description" here -- Should we have the category description reference a file even if it is not being used right now in case we want to use it later? -- This parameter is actually already in place currently, but with no values filled in.

10. 👍 Rename the type "legacy" to "hazards" or something similar

Other things to note: If we are having a discussion about adding a description parameter to sources & categories, we should also discuss adding a description parameter to measurements (pictured in diagram as well). These might not be used now but it's good to have them in place for the future.

[minniewong]

let's discuss the above two comments when @rboller gets back

[rboller]

Once descriptions are broken out on a per-layer basis, let's send over a preview to JPL for their further review.