Esri / street-data-processing-tools

ArcGIS geoprocessing toolbox containing tools to create and build a network dataset from commercial street data.
Apache License 2.0
36 stars 16 forks source link

There were too many records in the sign info table for ID {float(id)}, which appears " TypeError: float() argument must be a string or a number, not 'tuple' #56

Closed kvdmdi2 closed 6 months ago

kvdmdi2 commented 6 months ago

I was trying to build a route network based on TomTom when I got this error :

Traceback (most recent call last): File "", line 466, in execute File "M:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 435, in process_multinet_data self._populate_signposts_and_signposts_streets() File "M:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 148, in wrapper return inner_func() File "M:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 141, in inner_func return_val = func(*args, kwargs) File "M:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 1678, in _populate_signposts_and_signposts_streets raise ex File "M:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 1622, in _populate_signposts_and_signposts_streets f"**There were too many records in the sign info table for ID {float(id)}, which appears " TypeError: float() argument must be a string or a number, not 'tuple'**** Failed to execute (ProcessMultiNet).

Schermafbeelding 2024-03-01 092842

any ideas ?

thx

Koen

mmorang commented 6 months ago

Hello @kvdmdi2. I'm investigating this issue. I understand the technical reason this error is happening in the code, but I don't understand what data configuration would lead to the problem in the first place. Would it be possible for you to share your data with me so I can debug it? I can provide a secure file transfer option if needed.

kvdmdi2 commented 6 months ago

seems to be the arcgis pro version. version 2.9 works with warnings, version 3.2.1 gives error Screenshot 2024-03-04 111101

mmorang commented 6 months ago

Based on what the code is doing, I suspect it's related to the version of the pandas package included in the python environment included in ArcGIS Pro. Pro 3.2 has a newer version of pandas. We ran into some other issues elsewhere when this pandas version change occurred. I'm not really sure specifically what's going on here, and without being able to reproduce the problem, I can't really fix it. Are you able to share your data with me?

mmorang commented 6 months ago

Actually, now that I think about it, I suspect I know what's happening.

Can you try the version of for-ArcGIS-Pro\Process_MultiNet.py in this branch? https://github.com/Esri/street-data-processing-tools/tree/fix-pandas-groupby

In case you're not familiar with checking out GitHub branches, here's what you can do:

  1. Go to https://github.com/Esri/street-data-processing-tools/blob/fix-pandas-groupby/for-ArcGIS-Pro/Process_MultiNet.py
  2. Click the Download raw file button image
  3. Save the Process_MultiNet.py file over the existing one in the location you downloaded the tool previously.
  4. Close and reopen Pro if you have it open. (If you skip this step, Pro won't pick up the changes to the Python script.)
  5. Re-run the tool. I think it should work this time.

(Please test using Pro 3.2, as we already know it works in 2.9.)

kvdmdi2 commented 6 months ago

Hi, It runs 2 steps more, but hen it crashed again Schermafbeelding 2024-03-05 072944

Start Time: 05 March 2024 07:14:03 Validating inputs... Inputs validated successfully. Creating output geodatabase and feature dataset at C:\ArcGis\TT_2023_12_Route_Network_2024_03_05_BE2.gdb\Routing... Reading and grouping restrictions table... Reading and grouping maneuver paths table... Copying input network geometry feature class to target feature dataset... Populating restriction and historical traffic fields in streets... Reading and indexing Streets table... Creating turn feature class... Populating turn feature class... Creating road forks table... Populating road forks table... Creating Signposts feature class... Creating Signposts_Streets table... Populating Signposts feature class and Signposts_Streets table... There were too many records in the sign info table for ID 10561001093079.0, which appears in the sign path table. The signpost (ObjectID 4) will be truncated. There were too many records in the sign info table for ID 10561001105482.0, which appears in the sign path table. The signpost (ObjectID 10) will be truncated. There were too many records in the sign info table for ID 10561001148922.0, which appears in the sign path table. The signpost (ObjectID 138) will be truncated. There were too many records in the sign info table for ID 10561001148946.0, which appears in the sign path table. The signpost (ObjectID 139) will be truncated. There were too many records in the sign info table for ID 10561001152696.0, which appears in the sign path table. The signpost (ObjectID 153) will be truncated. There were too many records in the sign info table for ID 10561001173891.0, which appears in the sign path table. The signpost (ObjectID 250) will be truncated. There were too many records in the sign info table for ID 10561001182713.0, which appears in the sign path table. The signpost (ObjectID 328) will be truncated. There were too many records in the sign info table for ID 10561001195408.0, which appears in the sign path table. The signpost (ObjectID 431) will be truncated. There were too many records in the sign info table for ID 10561001195822.0, which appears in the sign path table. The signpost (ObjectID 435) will be truncated. There were too many records in the sign info table for ID 10561001206969.0, which appears in the sign path table. The signpost (ObjectID 500) will be truncated. There were too many records in the sign info table for ID 10561001309701.0, which appears in the sign path table. The signpost (ObjectID 787) will be truncated. There were too many records in the sign info table for ID 10561001326348.0, which appears in the sign path table. The signpost (ObjectID 805) will be truncated. There were too many records in the sign info table for ID 10561001349429.0, which appears in the sign path table. The signpost (ObjectID 841) will be truncated. There were too many records in the sign info table for ID 10561002095687.0, which appears in the sign path table. The signpost (ObjectID 1010) will be truncated. There were too many records in the sign info table for ID 10561002371300.0, which appears in the sign path table. The signpost (ObjectID 1123) will be truncated. There were too many records in the sign info table for ID 10561002633945.0, which appears in the sign path table. The signpost (ObjectID 1194) will be truncated. There were too many records in the sign info table for ID 10561002742921.0, which appears in the sign path table. The signpost (ObjectID 1220) will be truncated. There were too many records in the sign info table for ID 10561002760812.0, which appears in the sign path table. The signpost (ObjectID 1239) will be truncated. There were too many records in the sign info table for ID 10561003047471.0, which appears in the sign path table. The signpost (ObjectID 1416) will be truncated. There were too many records in the sign info table for ID 10561003087010.0, which appears in the sign path table. The signpost (ObjectID 1484) will be truncated. There were too many records in the sign info table for ID 10561003092541.0, which appears in the sign path table. The signpost (ObjectID 1515) will be truncated. There were too many records in the sign info table for ID 10561003098703.0, which appears in the sign path table. The signpost (ObjectID 1537) will be truncated. There were too many records in the sign info table for ID 10561003176218.0, which appears in the sign path table. The signpost (ObjectID 1642) will be truncated. There were too many records in the sign info table for ID 10561003216305.0, which appears in the sign path table. The signpost (ObjectID 1718) will be truncated. There were too many records in the sign info table for ID 10561004120390.0, which appears in the sign path table. The signpost (ObjectID 2040) will be truncated. There were too many records in the sign info table for ID 10561004227121.0, which appears in the sign path table. The signpost (ObjectID 2085) will be truncated. There were too many records in the sign info table for ID 10561004439285.0, which appears in the sign path table. The signpost (ObjectID 2120) will be truncated. There were too many records in the sign info table for ID 10561004602555.0, which appears in the sign path table. The signpost (ObjectID 2309) will be truncated. There were too many records in the sign info table for ID 10561004611934.0, which appears in the sign path table. The signpost (ObjectID 2359) will be truncated. There were too many records in the sign info table for ID 10561004613913.0, which appears in the sign path table. The signpost (ObjectID 2402) will be truncated. There were too many records in the sign info table for ID 10561004617501.0, which appears in the sign path table. The signpost (ObjectID 2439) will be truncated. There were too many records in the sign info table for ID 10561004650345.0, which appears in the sign path table. The signpost (ObjectID 2458) will be truncated. There were too many records in the sign info table for ID 10561004675301.0, which appears in the sign path table. The signpost (ObjectID 2486) will be truncated. There were too many records in the sign info table for ID 10561004681859.0, which appears in the sign path table. The signpost (ObjectID 2526) will be truncated. There were too many records in the sign info table for ID 10561004682488.0, which appears in the sign path table. The signpost (ObjectID 2585) will be truncated. There were too many records in the sign info table for ID 10561074529032.0, which appears in the sign path table. The signpost (ObjectID 2629) will be truncated. There were too many records in the sign info table for ID 10561075052485.0, which appears in the sign path table. The signpost (ObjectID 2668) will be truncated. There were too many records in the sign info table for ID 10561078965357.0, which appears in the sign path table. The signpost (ObjectID 2686) will be truncated. Creating and populating Streets_DailyProfiles table... Creating and populating DailyProfiles table... Traceback (most recent call last): File "", line 466, in execute File "C:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 440, in process_multinet_data self._create_and_populate_profiles_table() File "C:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 148, in wrapper return inner_func() File "C:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 141, in inner_func return_val = func(*args, **kwargs) File "C:\street-data-processing-tools-pro-v1.0.3\street-data-processing-tools-pro-v1.0.3\for-ArcGIS-Pro\Process_MultiNet.py", line 749, in _create_and_populate_profiles_table cur.insertRow(new_row) TypeError: value #0 - unsupported type: tuple Failed to execute (ProcessMultiNet). Failed at 05 March 2024 07:24:47 (Elapsed Time: 10 minutes 44 seconds)

mmorang commented 6 months ago

Okay, looks like another place in the code with that same pandas issue. I've updated the branch again. Please repeat your test with the updated code (same procedure as above). Thanks for your patience.

kvdmdi2 commented 6 months ago

No worries. I'm glad to be of use. The last changes did the trick !

image

mmorang commented 6 months ago

Okay, great! I will merge those changes into the main branch and close this issue.

mmorang commented 6 months ago

Fixed in https://github.com/Esri/street-data-processing-tools/pull/57