DOI-USGS / gems-tools-pro

GeMS Tools for ArcGIS Pro
Creative Commons Zero v1.0 Universal
45 stars 15 forks source link

Validate database inconsistently flagging missing or mis-defined fields (Rule 3.1)? #94

Closed tfelger closed 8 months ago

tfelger commented 9 months ago

I have a GeMS database that includes an OrientationPoints FC. I created it in 2019 so it contains field StationID instead of StationsID. The validate database tool flags StationID as an extension to the GeMS schema, but does not flag StationsID as a missing field. I also have CartographicPoints and MapUnitLabelPoints FCs, both of which are flagged as extensions to GeMS. The MapUnitLabelPoints FC lacks a MapUnitLabelPoints_ID field, and this is flagged by rule 3.1 as a missing or mis-defined field. Cartographic Points has a CartographicLines_ID (instead of a CartographicPoints_ID field), but it isn't flagged as either a schema extension or a missing or mis-defined field.

Can you please clarify what rule 3.1 considers when looking for missing or mis-defined fields? Maybe it's working as it should and I'm just not understanding what it's looking for?

Thanks!

ethoms-usgs commented 9 months ago

Try the latest version (not latest release).

Yes, there were a couple subtle evaluations going on that didn't consistently report suggested fields or _ID fields.

Now all "Cartographic" and "LabelPoints" feature classes need to have at least a Type, Symbol, Label, DataSourceID, and _ID field in them. Other fields listed in the GeMS definition but missing from the table itself are listed as recommended fields in Warnings.

CartographicPoints used to get classified as a "GenericPoints" feature class, where all fields in the GeMS definition are listed as optional.

Finally, the tool was happy with simply finding a field ending in _ID, but it wasn't checking to make sure that field was named after the table.

I mocked up a gdb with your some of your field names (and a MapUnitPolysLabelPoints fc with no fields at all) and got these results

MISSING MapUnitPolysLabelPoints missing field MapUnitPolysLabelPoints_ID CartographicPoints missing field CartographicPoints_ID EXTENSIONS Fields OrientationPoints, StationID OrientationPoints, ExternalSourceID CartographicPoints, CartographicLines_ID WARNINGS Recommended fields that are missing OrientationPoints field StationsID MapUnitPolysLabelPoints field Type MapUnitPolysLabelPoints field Symbol MapUnitPolysLabelPoints field Label MapUnitPolysLabelPoints field LocationConfidenceMeters MapUnitPolysLabelPoints field PlotAtScale MapUnitPolysLabelPoints field StationsID MapUnitPolysLabelPoints field MapUnit MapUnitPolysLabelPoints field LocationSourceID MapUnitPolysLabelPoints field DataSourceID MapUnitPolysLabelPoints field Notes CartographicPoints field LocationConfidenceMeters CartographicPoints field PlotAtScale CartographicPoints field StationsID CartographicPoints field MapUnit CartographicPoints field LocationSourceID

Let me know if this version works for you and I will publish it as a new release

tfelger commented 9 months ago

I tried your updated script and am getting this error now: image

ethoms-usgs commented 9 months ago

If the cause of that error is what I think it is, I don't know how it didn't happen for me. Try the latest version. Only GeMS_Definition.py is new.

tfelger commented 9 months ago

The script ran without error, but didn't quite produce the output I was expecting. It does flag MapUnitLabelPoints_ID as missing (per 3.1), and StationsID as a recommended missing field (took me a minute to find that section!). But it's not flagging the CartographicLines_ID as an extension. Here's what the extensions field list looks like: GlobalID is the only extension for CartoPoints.

image

ethoms-usgs commented 8 months ago

Hmm, it does for me image Could you send me your gdb?

tfelger commented 8 months ago

Hi Evan, it turns out that the field name is correct (CartographicPoints_ID) but the field alias somehow was set to CartographicLines_ID. Mystery solved - the script is working as you said it would! Thanks!

ethoms-usgs commented 8 months ago

Great! You're welcome. I don't think it would be too hard for the tool to check each field for name-alias mismatch. I will put it in the queue.