New Validate tool error (v2.11.2): potential error writing to html file?

[alwunder]

Testing the newest version on an incomplete database (complete except for DMU, Glossary, and DataSources table) and get the following error:

Looking at level 3 compliance
3.1 Table and field definitions conform to GeMS schema
3.2 All MapUnitPolys and ContactsAndFaults based feature classes obey Level 3 topology rules: 
        no ContactsAndFaults overlaps, self-overlaps, or self-intersections.
3.3 No missing required values
3.4 No missing terms in Glossary
3.5 No unnecessary terms in Glossary
3.6 No missing sources in DataSources
3.7 No unnecessary sources in DataSources
3.8 No map units without entries in DescriptionOfMapUnits
3.9 No unnecessary map units in DescriptionOfMapUnits
3.10 HierarchyKey values in DescriptionOfMapUnits are unique and well formed
Traceback (most recent call last):
  File "C:\GIS\TOOLS\GeMS\gems-tools-pro-2.11\Scripts\GeMS_ValidateDatabase.py", line 1549, in <module>
    rule2_1_results = rule2_1(db_dict, is_gpkg)
  File "C:\GIS\TOOLS\GeMS\gems-tools-pro-2.11\Scripts\GeMS_ValidateDatabase.py", line 1477, in main
    val["errors_path"] = str(workdir / f"{gdb_name}-ValidationErrors.html")
ValueError: too many values to unpack (expected 2)
Failed script Validate Database...
Failed to execute (ValidateDatabase).
Failed at Monday, August 28, 2023 2:39:15 PM (Elapsed Time: 1 minutes 34 seconds)

It seems at least this time I am getting errors on valid lines...

I also ran the tool on a finished database that passed previous validations and the tool runs to completion without error.

UPDATE: Adding commented lines to the code causes the error to move around, so I am relatively confident that the script is running form the most recent file and not caching.

[ethoms-usgs]

Good point. You should be able to use the tool on incomplete databases. I am surprised you got that far because when I deleted those three tables from a gdb, the tool bombed out much earlier. Anyway, I add some tests and got it to run to completion. In the cases where those tables are missing, many rules are irrelevant (I think? please let me know) and so the rule is not even checked.

To try this out, download from the green Code button, not the latest release. I will tag and post a new release if/when we get this worked out.

[alwunder]

Well, the tables are not deleted, they are simply empty. Not sure if that makes a difference. I will try to download again as you suggested. It'll get worked out!

[ethoms-usgs]

Ok, that would. I will need to work on that.

[alwunder]

OK, I just tested the latest version (2.11.3; copy pasted code directly from the Code page) again on the same incomplete database, and it finished, no problem! It recognized the empty tables and reported the missing entries in the DMU and Glossary tables, as expected:

2.4 MapUnits missing from DMU. Only one reference to each missing unit is cited
Mm, field MapUnit, table MapUnitPolys
Mfp, field MapUnit, table MapUnitPolys
Msl, field MapUnit, table MapUnitPolys
Mw, field MapUnit, table MapUnitPolys
Sbr, field MapUnit, table MapUnitPolys
Os, field MapUnit, table MapUnitPolys
Olcy, field MapUnit, table MapUnitPolys
2.6 Missing terms in Glossary. Only one reference to each missing term is cited
table ContactsAndFaults, field IdentityConfidence, certain
table ContactsAndFaults, field ExistenceConfidence, unspecified
table ContactsAndFaults, field IdentityConfidence, unspecified
table ContactsAndFaults, field ExistenceConfidence, certain
table ContactsAndFaults, field Type, contact approximate
table ContactsAndFaults, field Type, contact accurate
table MapUnitPolys, field IdentityConfidence, certain
table ContactsAndFaults, field Type, map neatline
2.8 Missing DataSources entries. Only one reference to each missing entry is cited
MapUnitPolys, field DataSourceID, TGSGQM_087NW_002
ContactsAndFaults, field DataSourceID, USGSMAPINDEX
ContactsAndFaults, field DataSourceID, TGSGQM_087NW_002

However, it is still tripping up on the Anno classes:

2.1 Missing required elements
Feature class ContactsAndFaultsAnno
2.2 Missing or mis-defined fields
MapUnitPolysAnno missing field MapUnit
MapUnitPolysAnno missing field IdentityConfidence
MapUnitPolysAnno missing field Label
MapUnitPolysAnno missing field Symbol
MapUnitPolysAnno missing field DataSourceID
MapUnitPolysAnno missing field MapUnitPolysAnno_ID

Getting closer now!!

[ethoms-usgs]

Is MapUnitPolysAnno an Arc-generated annotation class? The tool currently excludes annotation based on the 'featureType' property of a Describe object so it should be getting tagged as "Annotation"

[alwunder]

Yes, it is generated in ArcGIS Pro from the Maplex label engine, converted to annotation. I ran a bit of code in ArcGIS Pro Python window just to check:

import arcpy

# Get the layer as a parameter and describe it.

layerString = "MapUnitPolysAnno"
desc = arcpy.Describe(layerString)

# Print selected layer and describe object properties

print("Name: {}".format(desc.name))
if hasattr(desc, "layer"):
    print("Layer name: {}".format(desc.layer.name))
    print("Layer data source: {}".format(desc.layer.catalogPath))
    print("Feature type: {}".format(desc.layer.featureType))
else:
    print("Layer name: {}".format(desc.name))
    print("Layer data source: {}".format(desc.catalogPath))
    print("Feature type: {}".format(desc.featureType))

Name: MapUnitPolysAnno
Layer name: MapUnitPolysAnno
Layer data source: C:\GIS\PROJECTS\FY2022 STATEMAP Project\BelvidereTN_GeologicMap\BelvidereTN_GeologicMap.gdb\CartographicElements\MapUnitPolysAnno
Feature type: Annotation

[ethoms-usgs]

Ok, could you double-check that the file GeMS_utilityFunctions.py in Scripts looks like this?

In particular, def gdb_object_dict(gdb_path) starting on line 282?

[alwunder]

Alright! I double checked and refreshed the GeMS_utilityFunctions.py and it is now working: no trip-ups on the Annotation and the results are as expected, giving me lists of missing items so I can build my tables.

Awesome work, Evan! Thank you!!