wgs-asteely commented 8 months ago

Everything works great up to step 3.9, as below. I've run the tool successfully on several other gdbs. Not sure why this one is failing. One possibility is that the entries in MapUnit are kinda long...several are 8 characters, e.g. "Mv(gssc)". That's honestly the only thing I can see is different about this compared to other succesful runs.

Start Time: Friday, March 15, 2024 12:25:18 PM This version of the tool is up to date: GeMS_ValidateDatabase.py, version of 11/28/2023

{omitted for clarity]

3.9 No unnecessary map units in DescriptionOfMapUnits Traceback (most recent call last): File "J:\geologic_mapping\methods_tools_instructions\GeMS\gems-tools-pro-2.12.9\Scripts\GeMS_ValidateDatabase.py", line 1716, in dmu_path = db_dict["DescriptionOfMapUnits"]["catalogPath"] File "J:\geologic_mapping\methods_tools_instructions\GeMS\gems-tools-pro-2.12.9\Scripts\GeMS_ValidateDatabase.py", line 1624, in main for f in dmu_fields: File "J:\geologic_mapping\methods_tools_instructions\GeMS\gems-tools-pro-2.12.9\Scripts\GeMS_ValidateDatabase.py", line 447, in check_map_units

find all tables that have MapUnit-y field names

IndexError: list index out of range Failed script Validate Database... Failed to execute (ValidateDatabase). Failed at Friday, March 15, 2024 12:25:58 PM (Elapsed Time: 40.23 seconds)

screenshot of DMU table attached

ethoms-usgs commented 8 months ago

Ok, I don't think it's the length. MapUnit is built by default in all tables with a length of 10 characters, and I don't think there is a reason it can't be modified to be longer. I also think parentheses should be ok, so I don't know off-hand what's triggering the error. Have you used those before?

wgs-asteely commented 8 months ago

I've used the tool on similar types of MapUnit names. These were just a bit longer than others we use so I thought that might be it. Though I also considered they were under the length limit...so I'm puzzled too...

From: Evan Thoms @.> Sent: Monday, March 18, 2024 10:36 AM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

Ok, I don't think it's the length. MapUnit is built by default in all tables with a length of 10 characters, and I don't think there is a reason it can't be modified to be longer. I also think parentheses should be ok, so I don't know off-hand what's triggering the error. Have you used those before?

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2004539506, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKMLAKBIVANGAP5CRMLYY4QYPAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBUGUZTSNJQGY. You are receiving this because you authored the thread.Message ID: @.***>

ethoms-usgs commented 8 months ago

Have you checked both fields for leading or trailing empty spaces?

wgs-asteely commented 8 months ago

Hey Evan, I've run the white space tool and done a ctrl-f on every MapUnit field in the gdb for a white space character. None.

If it's not length, and not a white space, what does that leave us with? I assume that if it did find there was a DMU entry for which there was no associated MapUnit feature in any of the FCs, then it would just return that as something to look in to, not fail the script. Is that a good assumption, or is this failure telling me something about a mismatch between DMU map units and actual MapUnits?

Thanks so much!

Alexander N. Steely, PhD, LG (he/him) Assistant State Geologist Assistant Director of Geologic Hazards and Mapping Washington Geological Survey Cell: 360-999-0115 @.**@.>

From: Evan Thoms @.> Sent: Monday, March 18, 2024 1:43 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

Have you checked both fields for leading or trailing empty spaces?

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2004951250, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKO3ZQAC4BHLWD35PETYY5GV5AVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBUHE2TCMRVGA. You are receiving this because you authored the thread.Message ID: @.***>

ethoms-usgs commented 8 months ago

Sorry, Alex. I didn't look super closely at the error message at first, but the report suggests to me that Arc is running a cached version of the script and may not be the most recent. See in the error message how it says that the error is happening when it hits line 447? That line is a comment; it can't trigger an error. This can happen when the application does not know about a new version that should replace a cached version. Can you close everything down and then try again?

wgs-asteely commented 8 months ago

Hey Evan, Fascinating. So I have restarted my machine a bunch of times during this. And I just downloaded and used the 12.13 version instead of 12.9 that I had been. No change. Still errors out at the same place with the same error.

So I deleted the default gdb and default toolbox. And changed locations for all the gems tools. Still same error. Very puzzling...

From: Evan Thoms @.> Sent: Monday, March 18, 2024 6:13 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

Sorry, Alex. I didn't look super closely at the error message at first, but the report suggests to me that Arc is running a cached version of the script and may not be the most recent. See in the error message how it says that the error is happening when it hits line 447? That line is a comment; it can't trigger an error. This can happen when the application does not know about a new version that should replace a cached version. Can you close everything down and then try again?

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2005551846, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKK25LVD7JCY35OB573YY6GLLAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBVGU2TCOBUGY. You are receiving this because you authored the thread.Message ID: @.***>

ethoms-usgs commented 8 months ago

Ok. Thanks for trying that. Can you send me the gdb?

wgs-asteely commented 8 months ago

Sure. Thanks so much for taking a look! I'm attaching here... KittitasEastKittitas_geologicmap_14MAR2024.gdb.zip

ethoms-usgs commented 8 months ago

Odd, I just ran the tool on it and there were no code errors. When you downloaded the new toolbox, did you completely replace the older folder? Also, try downloading to a new empty folder in a different location and try it from there.

wgs-asteely commented 8 months ago

Hey Evan, So strange. We'll I'm glad it is error free, but I do need to convince my computer it is too.

I have replaced the older tool folder. Renamed the gdb. Started from a blank map project. Disconnected from our vpn and stored everything locally. Deleted again everything in my default Arc locations (I think). And still the same issue. Aside from a fresh ArcPro install I'm not sure what else to do.

You said that this can happen when Arc is looking to a cached version of the GeMS_validate script...do you know where that might live?

Thanks, Alex

From: Evan Thoms @.> Sent: Tuesday, March 19, 2024 2:33 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

Odd, I just ran the tool on it and there were no code errors. When you downloaded the new toolbox, did you completely replace the older folder? Also, try downloading to a new empty folder in a different location and try it from there.

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2008172608, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKME724AGHXEXO646OLYZCVKLAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBYGE3TENRQHA. You are receiving this because you authored the thread.Message ID: @.***>

wynaut commented 8 months ago

I am also not able to replicate the error... Interestingly, I'm not even able to trigger rule 3.9 on purpose on this one -- nonsense units added to the DMU get totally skipped over! It's also not flagging the null values in DescriptionSourceID (Ql, hy).

I thought it might have something to do with having 'geologicmap' in the gdb name (since that's also the feature dataset's name), or with Windows' path length limit. But no...

ethoms-usgs commented 8 months ago

Should have looked at ESRI earlier:

https://community.esri.com/t5/arcgis-pro-questions/scripting-tool-does-not-return-line-error/td-p/1262701

Alex, what version of Pro are you on? They say this bug has been addressed in 3.2

ethoms-usgs commented 8 months ago

@wynaut, uh oh, really? I get validation errors when I add an unused unit and it catches null values in _ID

wgs-asteely commented 8 months ago

Hmm. Very strange. We're all on 2.9 still and there is no real plan to get to 3.x anytime soon. So...there is an error, but it's just not at the location it says it is? As people in that Esri thread said...that's not very helpful :-/

From: Evan Thoms @.> Sent: Wednesday, March 20, 2024 2:17 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

Should have looked at ESRI earlier:

https://community.esri.com/t5/arcgis-pro-questions/scripting-tool-does-not-return-line-error/td-p/1262701

Alex, what version of Pro are you on? They say this bug has been addressed in 3.2

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2010654686, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKK5LP5HJ3SYLELCPDLYZH4FPAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJQGY2TINRYGY. You are receiving this because you authored the thread.Message ID: @.***>

wgs-asteely commented 8 months ago

Evan, I double checked that I can run the script just fine on other gdbs. So, at least to my computer, there is something different and problematic about the KittitasEastKittitas gdb.

From: Evan Thoms @.> Sent: Wednesday, March 20, 2024 2:17 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

Should have looked at ESRI earlier:

https://community.esri.com/t5/arcgis-pro-questions/scripting-tool-does-not-return-line-error/td-p/1262701

Alex, what version of Pro are you on? They say this bug has been addressed in 3.2

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2010654686, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKK5LP5HJ3SYLELCPDLYZH4FPAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJQGY2TINRYGY. You are receiving this because you authored the thread.Message ID: @.***>

ethoms-usgs commented 8 months ago

But the gdb is fine for me. Weird.

Try this. Paste the text below into a file with any name and a .py extension (I called it validate_this.py)

import arcpy

arcpy.ImportToolbox(r"C:\AAA\gems\gitspace\gems-tools-pro\GeMS_Tools.tbx")
arcpy.GEMS.ValidateDatabase(
    r"C:\AAA\gems\testing\kittitas\KittitasEastKittitas_geologicmap_14MAR2024.gdb",
    r"C:\AAA\gems\testing\kittitas",
    None,
    None,
    False,
    True,
    False,
    False,
    True,
    True)

Change the parameters as you need them. You will have to follow along with the parameter form to figure out what each one controls. Be sure to leave the r in front of the file paths.

Open the Python Command Prompt under ArcGIS at Windows Start

and cd to the folder where you saved that file. Then type

python validate_this.py

(arcgispro-py3) C:\AAA\gems\testing\kittitas>python validate_this.py

My guess is that the code/error mismatch is happening in Pro, with how Pro manages python. But I suspect that the actual installation of python is fine.

wgs-asteely commented 8 months ago

Suspicion denied? Same error in same place. This is too weird…

(arcgispro-py3) C:\Users\asty490\Desktop\HL_BH_KEK_gems>python thoms_test.py This version of the tool is up to date: GeMS_ValidateDatabase.py, version of 11/28/2023 Looking at level 2 compliance Rule 2.1 - Has required elements: nonspatial tables DataSources, DescriptionOfMapUnits, GeoMaterialDict; feature dataset GeologicMap with feature classes ContactsAndFaults and MapUnitPolys Rule 2.2 - Required fields within required elements are present and correctly defined 2.3 All MapUnitPolys and ContactsAndFaults based feature classes obey Level 2 topology rules: no internal gaps or overlaps in MapUnitPolys, boundaries of MapUnitPolys are covered by ContactsAndFaults Topology check was skipped 2.4 All map units in MapUnitPolys have entries in DescriptionOfMapUnits table 2.5 No duplicate MapUnit values in DescriptionOfMapUnits table 2.6 Certain field values within required elements have entries in Glossary table 2.7 No duplicate Term values in Glossary table 2.8 All xxxSourceID values in required elements have entries in DataSources table 2.9 No duplicate DataSources_ID values in DataSources table Looking at level 3 compliance 3.1 Table and field definitions conform to GeMS schema 3.2 All MapUnitPolys and ContactsAndFaults based feature classes obey Level 3 topology rules: no overlaps, self-overlaps, or self-intersections in ContactsAndFaults. 3.3 No missing required values 3.4 No missing terms in Glossary 3.5 No unnecessary terms in Glossary 3.6 No missing sources in DataSources 3.7 No unnecessary sources in DataSources 3.8 No map units without entries in DescriptionOfMapUnits 3.9 No unnecessary map units in DescriptionOfMapUnits Traceback (most recent call last): File "thoms_test.py", line 14, in True) File "C:\Users\asty490\Desktop\HL_BH_KEK_gems\gems-tools-pro-2.12.13\GeMS_Tools.tbx", line 1115, in ValidateDatabase File "C:\Users\asty490\Desktop\HL_BH_KEK_gems\gems-tools-pro-2.12.13\GeMS_Tools.tbx", line 1112, in ValidateDatabase File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing_base.py", line 512, in return lambda args: val(gp_fixargs(args, True)) arcgisscripting.ExecuteError: Traceback (most recent call last): File "C:\Users\asty490\Desktop\HL_BH_KEK_gems\gems-tools-pro-2.12.13\Scripts\GeMS_ValidateDatabase.py", line 1716, in dmu_path = db_dict["DescriptionOfMapUnits"]["catalogPath"] File "C:\Users\asty490\Desktop\HL_BH_KEK_gems\gems-tools-pro-2.12.13\Scripts\GeMS_ValidateDatabase.py", line 1624, in main for f in dmu_fields: File "C:\Users\asty490\Desktop\HL_BH_KEK_gems\gems-tools-pro-2.12.13\Scripts\GeMS_ValidateDatabase.py", line 447, in check_map_units

find all tables that have MapUnit-y field names

IndexError: list index out of range

Failed to execute (ValidateDatabase).

From: Evan Thoms @.> Sent: Wednesday, March 20, 2024 3:21 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

But the gdb is fine for me. Weird.

Try this. Paste the text below into a file with any name and a .py extension (I called it validate_this.py)

import arcpy

arcpy.ImportToolbox(r"C:\AAA\gems\gitspace\gems-tools-pro\GeMS_Tools.tbx")

arcpy.GEMS.ValidateDatabase(

r"C:\AAA\gems\testing\kittitas\KittitasEastKittitas_geologicmap_14MAR2024.gdb",

r"C:\AAA\gems\testing\kittitas",

None,

None,

False,

True,

False,

False,

True,

True)

Change the parameters as you need them. You will have to follow along with the parameter form to figure out what each one controls. Be sure to leave the r in front of the file paths.

Open the Python Command Prompt under ArcGIS at Windows Start image.png (view on web)https://github.com/DOI-USGS/gems-tools-pro/assets/5376315/1db12b29-3383-48f6-8a1a-b55e04e097d5

and cd to the folder where you saved that file. Then type

python validate_this.py

(arcgispro-py3) C:\AAA\gems\testing\kittitas>python validate_this.py

My guess is that the code/error mismatch is happening in Pro, with how Pro manages python. But I suspect that the actual installation of python is fine.

— Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2010749521, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKKYI4IA47BXDXWLJ4TYZIDV7AVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJQG42DSNJSGE. You are receiving this because you authored the thread.Message ID: @.***>

wgs-asteely commented 8 months ago

Hi Evan, So I just tried on a brand new computer with a fresh install of ArcPro and still had the same error. And I confirmed that I can validate other gdbs by my team just fine. Any further thoughts on this? Thanks, Alex

From: Evan Thoms @.> Sent: Wednesday, March 20, 2024 3:21 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

But the gdb is fine for me. Weird.

Try this. Paste the text below into a file with any name and a .py extension (I called it validate_this.py)

import arcpy

arcpy.ImportToolbox(r"C:\AAA\gems\gitspace\gems-tools-pro\GeMS_Tools.tbx")

arcpy.GEMS.ValidateDatabase(

r"C:\AAA\gems\testing\kittitas\KittitasEastKittitas_geologicmap_14MAR2024.gdb",

r"C:\AAA\gems\testing\kittitas",

None,

None,

False,

True,

False,

False,

True,

True)

Change the parameters as you need them. You will have to follow along with the parameter form to figure out what each one controls. Be sure to leave the r in front of the file paths.

Open the Python Command Prompt under ArcGIS at Windows Start image.png (view on web)https://github.com/DOI-USGS/gems-tools-pro/assets/5376315/1db12b29-3383-48f6-8a1a-b55e04e097d5

and cd to the folder where you saved that file. Then type

python validate_this.py

(arcgispro-py3) C:\AAA\gems\testing\kittitas>python validate_this.py

My guess is that the code/error mismatch is happening in Pro, with how Pro manages python. But I suspect that the actual installation of python is fine.

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2010749521, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKKYI4IA47BXDXWLJ4TYZIDV7AVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJQG42DSNJSGE. You are receiving this because you authored the thread.Message ID: @.***>

wynaut commented 8 months ago

@wynaut, uh oh, really? I get validation errors when I add an unused unit and it catches null values in _ID

@ethoms-usgs I tried experimenting further with my "unbreakable 3.9" finding, but it's not happening now -- restarting everything between Tuesday and today must have fixed it somehow... However, the second half of my comment remains true -- I thought that something weird was happening here due to the use of MapUnitOverlayPolys specifically, but I think it's more general. I'll open a separate issue.

ethoms-usgs commented 8 months ago

And what is the version of the new install of Pro on the new computer? If you are now at 3.2 and still get the same error, the only thing I can think of is to copy all feature classes and tables to a new gdb.

Or contact ESRI

wgs-asteely commented 8 months ago

It's still 2.9. I just thought that if it was a weird caching issues as you mentioned, that really starting from scratch would kill it. I also had a colleague validate it on their machine and it broke in the same spot. I'll try the copy bit...but I just cant figure out why it works fine on your machine but not mine...

From: Evan Thoms @.> Sent: Tuesday, March 26, 2024 2:27 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

And what is the version of the new install of Pro on the new computer? If you are now at 3.2 and still get the same error, the only thing I can think of is to copy all feature classes and tables to a new gdb.

Or contact ESRI

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2021502666, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKMLYMYHBWCIXH2YKH3Y2HRZFAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRRGUYDENRWGY. You are receiving this because you authored the thread.Message ID: @.***>

wgs-asteely commented 8 months ago

Evan, I have determined that the error is triggered when point FCs are in the gdb, but it runs fine when they are not there. The overlays and other line units will all pass fine, but whenever I add in a point FC it will fail. Even the simplest case, FossilPoints, which has a single point in it with a clearly correct MapUnit entry...and then when I delete the point FC, it runs fine.

From: Evan Thoms @.> Sent: Tuesday, March 26, 2024 2:27 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

And what is the version of the new install of Pro on the new computer? If you are now at 3.2 and still get the same error, the only thing I can think of is to copy all feature classes and tables to a new gdb.

Or contact ESRI

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2021502666, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKMLYMYHBWCIXH2YKH3Y2HRZFAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRRGUYDENRWGY. You are receiving this because you authored the thread.Message ID: @.***>

ethoms-usgs commented 8 months ago

Is this with a copy of the gdb created on your machine? Are you still getting the same error you first reported?

wgs-asteely commented 8 months ago

Same error. And I did make a copy and then deleted everything but CAF and MUP and then just kept adding things back and running the validate script to see when it would have the error.

From: Evan Thoms @.> Sent: Wednesday, March 27, 2024 2:51 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

Is this with a copy of the gdb created on your machine? Are you still getting the same error you first reported?

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2024052349, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKK2GC3FLM32MOHMNC3Y2M5NNAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRUGA2TEMZUHE. You are receiving this because you authored the thread.Message ID: @.***>

ethoms-usgs commented 8 months ago

Do the other gdbs on which the tool has run successfully have point feature classes? Is it just point feature classes that have a MapUnit field?

wgs-asteely commented 8 months ago

The other gdbs that work also have point FCs with MapUnit fields and they run fine. If I put the Stations FC into the funky gdb, then it validates just fine (so long as there are no other point FCs). So point FCs with MapUnit fields = breaking at step 3.9.

From: Evan Thoms @.> Sent: Wednesday, March 27, 2024 3:44 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

Do the other gdbs on which the tool has run successfully have point feature classes? Is it just point feature classes that have a MapUnit field?

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2024107928, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKIMAZLNMNXKSAZVG3LY2NDRVAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRUGEYDOOJSHA. You are receiving this because you authored the thread.Message ID: @.***>

ethoms-usgs commented 8 months ago

What a puzzle. But I still cannot reproduce the error and we still don't have an error with a line number that makes sense.

wgs-asteely commented 8 months ago

Hey Evan, This is bizarre. And I would totally just give up except that I cannot validate this gdb for statemap until we figure it out ☹

I thought about that esri link you sent indicating that wrong line codes can result from the arcpro toolbox and how it treats blank lines. This is definitely happening. I removed all the blank lines. Turns out I missed a couple, but that was good as I could watch which lines were returned as errors and see a progression. Then I removed all the comments and continued to see some progression. At present, this is what I think is actually causing the error. I’m attaching the altered python file too if that helps since my line numbers are now quite different than the original. Regardless of number, something appears to be happening in the check_map_units function…

@.***

Any thoughts?

Thanks so much and sorry it’s not resolving quickly… Alex

From: Evan Thoms @.> Sent: Wednesday, March 27, 2024 4:27 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

What a puzzle. But I still cannot reproduce the error and we still don't have an error with a line number that makes sense.

— Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2024147910, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKNV4NBWQPO63IIHNGTY2NIUNAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRUGE2DOOJRGA. You are receiving this because you authored the thread.Message ID: @.***>

wgs-asteely commented 8 months ago

And here is the python file

wgs-asteely commented 8 months ago

Hey Evan, Yesterday the error cropped up on a different gdb from a different user. So I dug in to that and found that it was again the same issue...the point FCs were causing the error. Long story short, I rebuilt the point FCs and reloaded the data back into them and it works fine. I can find no difference in the FC properties before or after the rebuild, nor any difference between those that do or don't work. But somehow deleting, recreating, and reloading the data from a copy of the gdb solves it. Ugh ArcPro, why?!?!

I'm sorry to have spent so much of your time on this. Though I cant say I understand at all what changed. But I guess now we know how to fix it and that's the important part.

Cheers (and I owe you a beer at a conference), Alex

From: Evan Thoms @.> Sent: Wednesday, March 27, 2024 4:27 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

What a puzzle. But I still cannot reproduce the error and we still don't have an error with a line number that makes sense.

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2024147910, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKNV4NBWQPO63IIHNGTY2NIUNAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRUGE2DOOJRGA. You are receiving this because you authored the thread.Message ID: @.***>

ethoms-usgs commented 7 months ago

That's very interesting. Do you still have a copy of the gdb that was causing the problem in the first place? If you do, could you run Repair Geometry - Repair Geometry (Data Management)—ArcGIS Pro | Documentationhttps://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/repair-geometry.htm on the offending feature classes? Perhaps that can be a tool that gets run from the Validate Database tool.

Although another interesting thing is that apparently Pro 3.0 can deal with the errors while 2.9 can't

-- Evan Thoms US Geological Survey, Alaska Science Center 4210 University Drive, Anchorage, AK 99508

From: wgs-asteely @.> Sent: Friday, March 29, 2024 11:30 AM To: DOI-USGS/gems-tools-pro @.> Cc: Thoms, Evan E @.>; Mention @.> Subject: [EXTERNAL] Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.

Hey Evan, Yesterday the error cropped up on a different gdb from a different user. So I dug in to that and found that it was again the same issue...the point FCs were causing the error. Long story short, I rebuilt the point FCs and reloaded the data back into them and it works fine. I can find no difference in the FC properties before or after the rebuild, nor any difference between those that do or don't work. But somehow deleting, recreating, and reloading the data from a copy of the gdb solves it. Ugh ArcPro, why?!?!

I'm sorry to have spent so much of your time on this. Though I cant say I understand at all what changed. But I guess now we know how to fix it and that's the important part.

Cheers (and I owe you a beer at a conference), Alex

From: Evan Thoms @.> Sent: Wednesday, March 27, 2024 4:27 PM To: DOI-USGS/gems-tools-pro @.> Cc: Steely, Alex (DNR) @.>; Author @.> Subject: Re: [DOI-USGS/gems-tools-pro] Validate gdb fails at step 3.9 (Issue #101)

External Email

What a puzzle. But I still cannot reproduce the error and we still don't have an error with a line number that makes sense.

- Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2024147910, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHACYKNV4NBWQPO63IIHNGTY2NIUNAVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRUGE2DOOJRGA. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHubhttps://github.com/DOI-USGS/gems-tools-pro/issues/101#issuecomment-2027657927, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABJASO6PJZG3HTGQEL4FR7DY2W6O7AVCNFSM6AAAAABEYR63E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRXGY2TOOJSG4. You are receiving this because you were mentioned.Message ID: @.***>

DOI-USGS / gems-tools-pro

Validate gdb fails at step 3.9 #101

find all tables that have MapUnit-y field names

find all tables that have MapUnit-y field names