New Validate tool error (v2.11.3): DMU table not found (Level 3)?

[alwunder]

Running the latest version of the tool on a database that previously passed validation throws a Level 3 error:

3.10 HierarchyKey values in DescriptionOfMapUnits are unique and well formed | FAIL DMU cannot be found. Rule not checked -- | --

However, the DMU table is included in the Validation.html report (I only included one row to keep this brief):

DescriptionOfMapUnits

OBJECTID | MapUnit | Name | FullName | Age | Description | HierarchyKey | ParagraphStyle | Label | Symbol | AreaFillRGB | AreaFillPatternDescription | DescriptionSourceID | GeoMaterial | GeoMaterialConfidence | DescriptionOfMapUnits_ID -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 1 | Qal | Alluvium | Alluvium | Holocene and Pleistocene (Quaternary) | Quartz sand, silt and clay, etc., etc... | 01 | DMUUnit1 | Qal | 20 | 255,255,204 | None | DAS001 | Alluvial sediment, mostly fine-grained | High | DMU001

I see in the code that the error messages are the same in parts 2.4, 2.5, 3.9, and 3.10 of the check, so the message that there is no DMU is erroneous in the latter checks. Perhaps the error message in could be updated to better explain why the script thinks the HK values are no good? Mismatch of count of MapUnits and HK values? Or...?

The stratigraphy on this map is very simple and the hierarchy key is [01, 02, 03, 04, 04-01, 04-02]. The last unit (04-02), however, is only shown in the cross section (which is not part of the database; it's a graphic on the published map only) and has a map unit of "None" and is not part of the inventory of the MapUnits in the "Occurrence of MapUnit in DMU..." section of the Validation Report. We included it in the database DMU because it is on the final published layout, but I see that this may prove problematic for the validation tool. I seem to recall that others have brought this up before (the map unit in cross section only problem) but I think it had to do with units that WERE in the database but in a CrossSectionX feature dataset.

This is a problem for us in other situations as well, where a stratigraphic column may identify a coal seam but the seam only shows up in outcrop too insignificant, completely mined out, or too sporadic to map and thus may not be included anywhere as a feature in the database. We have been reluctant to use the CMU feature dataset, but that may be the best way to solve the problem? There hasn't been much in the way of guidance or discussion of the use of the CMU that I recall. Perhaps this would be a good DMT topic.

I am not sure what else we could do to make a solution that is a consistent fix.

Thoughts?

[ethoms-usgs]

First, the DMU not being found was due to a typo. Fixed in the latest version (again, not the latest release just yet)

I will get back to you regarding stratigraphy.

[alwunder]

Got it. I copied the new Validation code directly and tested again, problem solved.

[ethoms-usgs]

Is the basic issue with your stratigraphy question that your DMU has entries for MapUnits which are not found in any MapUnit field anywhere else in the database? In that case, Rule 3.10 No unnecessary map units in DescriptionOfMapUnits would have to be relaxed. Is that right?

[alwunder]

Yes, exactly. However, the more I think about it, I'm not sure relaxing those rules is a good idea.

I know there are many, many geologic maps that are structured the same way as ours. And if GeMS is FULLY implemented on every single element on the published map, i.e., the CMU/stratigraphic column and cross section components are all included as features in the database, then it's no problem; every MapUnit element described will appear somewhere and the DMU test will find a match.

So, I think it's just a matter of adding some elements to the CMU feature dataset to satisfy Rule 3.10. Which brings me to the question of how features in those CMU classes are handled. I have digitized a stratigraphic column from one of our published maps as a test in the past, but didn't feel it was worthwhile to replicate the complexity of the published graphic. Including cross sections makes more sense to me as they provide the basis for potential 3D interpretive work, but we are still working on our methodology for that and are not prepared to include them in our databases at this time.

Also, our maps rarely have true "correlation" diagrams. But if we could use those CMU classes to simply house some representative "placeholder" features that store the MapUnit attribute so our databases pass, I guess that's what we need to do.

Does NGMDB use the CMU feature dataset for any specific purpose? Do the CMU features have to represent a published map graphic "at scale" and be arranged the same way with swatches, lines and labels? Is there any guidance on how to construct the CMU classes if you are not just copying from a published map?

[ethoms-usgs]

Yes, if you are willing to take the time to digitize some of the other graphics and attribute a MapUnit field, that would solve the problem. Dave, Ralph, and I agreed that we don't want to relax Rule 3.10, even though extra MapUnits in a DMU is, at least so far, simply not an issue in anyway. Going forward, if a map doesn't reach Level 2 because it fails Rule 3.10, the authors simply need to provide justification at submission. In the meantime, leaving the rule in place will hopefully encourage more complete databases.

As to CMUs, no, the NGMDB is not using them in anyway. The main argument for a digital version is to provide machine parseable data as opposed to a raster graphic, even though there is no standard schema for the organization of CMUs that would allow real geologic analysis to be done. If you have a graphic, yes, digitize that in a fake coordinate system using inches or mm. If not, no, I unfortunately have no specific guidance other than looking at others.

CMU's are (often) a representation of the topology of the units through time, which, if the database was robust enough, you could build programmatically. The Loop project has done this (see Fig. 9). Their 'Stratigraphy Graph' is pretty much a CMU.

[alwunder]

Yes, it's always a better problem to have more map units in the DMU than on the map and not the other way around...

The article you referenced is excellent; that is really impressive work those folks have done to deconstruct the 2D map and build the 2D-4D relationships necessary for 3D construction. And as you noted, I definitely think a well-formed GeMS DMU is robust enough to programmatically build a very simple CMU based on the HierarchyKey values, since that is in part what it was designed to capture when working "backwards" from a published map.

Back to the topic of the CMU feature dataset in GeMS. It seems to me that this is, perhaps, something that could be discussed further at a DMT (or DMT Lite) meeting or a question posed to the Listserv group. I'd definitely like to get others thoughts on if/how they use the GeMS CMU and what, if any, similarities there are on how they are designed and implemented.

Thanks again for your thoughts on this. I'll make the necessary changes to our databases to pass validation, and perhaps a more in-depth discussion of the CMU will spring up in the future.