search

DOI-USGS / gems-tools-pro

GeMS Tools for ArcGIS Pro
Creative Commons Zero v1.0 Universal
45 stars 15 forks source link

Toolbox doesn't have Fix String Values tool. #11

Closed tfelger closed 2 years ago

tfelger commented 3 years ago

I noticed that the Pro toolbox doesn't have the Fix String Values tool that is in the ArcMap toolbox. Will this be added at some point?

ethoms-usgs commented 3 years ago

Good catch. Wonder why I never added that before? Anyway, it is now in version 2.5. I did not test it on a gdb that I knew had errors so the successful run I got does not necessarily mean that the checks it does were free of bugs. Let us know how it works.

tfelger commented 3 years ago

Thanks Evan. I tried it out - the script runs, and correctly identifies leading and trailing spaces in string values, but doesn't actually fix them. Below is an example of the message I get for a line that has the word "granitic " (with a trailing space) in the notes field. I don't know how to interpret the error, since the Notes field is nullable. Thoughts?

MapUnitLines 1004 granitic failed to update row 1004. Perhaps a field is not nullable

ethoms-usgs commented 3 years ago

Tracey, I can't reproduce that error. For me, the tool is working except that I don't see the changes until I close the table, refresh the catalog view of the geodatabase and add the table again.

A confusing thing is that the error message you get is shown regardless of the actual exception when trying to write new values to the row in the table. For example, even if the reason a new value can't be written is because it exceeds the length of the field, you will still get that message.

tfelger commented 3 years ago

Hi Evan, I tried closing the table, refreshing, etc but that didn't help - the spaces are definitely still there. I thought maybe it was just a visual thing, but if I run the Validate Database tool, they get flagged again. Strange that it works for you, and that the error gets thrown regardless of the reason...

ethoms-usgs commented 3 years ago

Weird. Can you send me the gdb? Or a copy that is empty except for the table?

tfelger commented 3 years ago

Hi Evan, Sorry for the delay - I was out on A/L. I will send you a copy of the gdb and the validate reports to your USGS email. I don't want to post them here since the databases is not yet published.

ethoms-usgs commented 3 years ago

Looks like it's because the update event is triggering an attribute rule calculation that can't be completed. I commented out the try/except block and the scripted error message to see what python tells us and I got this

Traceback (most recent call last):
  File "C:\_AAA\gems\gitspace\gems-tools-pro\Scripts\GeMS_FixStrings_AGP2.py", line 62, in <module>
    fixTableStrings(r"C:\_AAA\gems\examples\Felger-LakeMead\NPSLakeMeadNRA_GeMS.gdb\GeologicMap\MapUnitLines")
  File "C:\_AAA\gems\gitspace\gems-tools-pro\Scripts\GeMS_FixStrings_AGP2.py", line 42, in fixTableStrings
    cursor.updateRow(row1)
RuntimeError: Failed to evaluate Arcade expression. [
Rule name: Calc _ID,
Triggering event: Update,
Class name: MapUnitLines,
GlobalID: ,
Arcade error: Field not found GlobalID,
Script line: 1]

Obviously, the table DOES have a GlobalID field, so that's weird. It's a similar problem to something reported on GeoNet https://community.esri.com/t5/arcgis-pro-questions/attribute-rules-causing-failures-in-field/td-p/203536

In the short term, a workaround could be to disable the rule, but I will look around some more

tfelger commented 3 years ago

Interesting - thanks. I'll try disabling/deleting the rule and let you know if that works.

tfelger commented 3 years ago

Disabling the attribute rule that calcs _ID from GlobalID solves the problem for my feature classes. But I have the same problem with the DescriptionOfMapUnits table, and it doesn't have any attribute rules associated with it. Can you work your magic and tell me what error python gives your for that?

ethoms-usgs commented 3 years ago

In that case, it turns out that there are empty spaces in the DescriptionOfMapUnits_ID field, which the tool tries to convert to nulls and comes up against the non-nullable property of the field.

It's a little tricky getting the name of the offending field because of the batch way in which the the updates are applied to each row, but that's the sort of messaging the tool needs. I will add it to the queue, but in the meantime is that another field you could add an attribute rule to? Is there a create event that can trigger a field calculation? I know we are finding out that attribute rules need to be disabled before the tool is run, but, aside from using this tool, if you have those rules in place during data entry, they could make sure that non-nullable fields are getting filled in correctly.

tfelger commented 3 years ago

Thanks for working your magic! I added an attribute rule to Calc the _ID in the tblDMU, disabled it and ran the Fix String tool again and it worked. I also had disabled the Calc _ID rule on my ContactsAndFaults, since it had found a value in that feature class with a leading/trailing space. For some reason it didn't fix that one when I ran the Fix String tool again. There is only one of those, so easy to fix by hand, but not sure why it didn't work for that one...