search

Esri / military-tools-geoprocessing-toolbox

military-tools-geoprocessing-toolbox is a collection of models, scripts, and tools for use in ArcGIS for Desktop and ArcGIS Pro. This toolbox is one component that is a part of Military Tools.
Apache License 2.0
33 stars 14 forks source link

Convert .tbx tools to .pyt: Conversion Tools #274

Closed lfunkhouser closed 6 years ago

lfunkhouser commented 6 years ago

Task to be done:

Validate Checklist for the following issues:

Issue https://github.com/Esri/military-tools-geoprocessing-toolbox/issues/262 has been broken down. The following are the tools from the conversion toolset:

Port the tools to pyt:

Tool Doc:

Acceptance Criteria: Target release: Military Tools 3.0 One toolbox Tests written and running for tools Tools documented User documentation complete and published Toolbox/doc included in the .zip Tools tested

csmoore commented 6 years ago

Starting working on tools in branch: https://github.com/Esri/military-tools-geoprocessing-toolbox/tree/csm/%23274-tool-stubs

BobBooth commented 6 years ago

Updated What's New and Military Tools home page in GitHub to indicate conversion to PYT format, in PR https://github.com/ArcGIS/arcgis-solutions-website/pull/6431

ACueva commented 6 years ago

Verified Conversion unit tests are running from within Jenkins: http://dev0002292:8080/job/GP%20Unit%20tests/job/ArcMap%20Test%20Runner/14/console Jenkins has been setup to run unit tests against ArcGIS Pro 2.1, ArcGIS Pro 2.2 daily, ArcMap 10.3.1 and ArcMap 10.6.1 Daily.

csmoore commented 6 years ago

The remaining conversion tools were adding in branch: https://github.com/Esri/military-tools-geoprocessing-toolbox/tree/csm/%23274-add-conv-tools & PR #283

@BobBooth - can you add the GP tool doc after this gets merged ? Thanks

BobBooth commented 6 years ago

Merging in doc from old tools to the PYT version.

BobBooth commented 6 years ago

Doc done in this PR: https://github.com/Esri/military-tools-geoprocessing-toolbox/pull/286

BobBooth commented 6 years ago

Added readme and contents updates for Military Tools.pyt in this PR: https://github.com/Esri/military-tools-geoprocessing-toolbox/pull/287

BobBooth commented 6 years ago

While looking at the tests for the Conversion tools, I observed that several were just looking for a count of features. I noticed that Table To Polygon was not being tested to output multiple polygons, even though the input table has groups for multiple polys. When I tested via the GUI, I found that the tool fails on the test data when it is given the "Name" field to group the polygons. Submitted this issue: https://github.com/Esri/military-tools-geoprocessing-toolbox/issues/289 Note: this is not just a problem with the PYT - the tool in the _arcmap TBX also fails this way.

dfoll commented 6 years ago

Removing G-Doc (CMS) verified tools in toolbox match the conversion toolset documentation

csmoore commented 6 years ago

Looking at automated tests for these now to see if sufficient or if test design should be captured. Also to see if PR #290 can be added without breaking tests.

ACueva commented 6 years ago

FYI @dfoll @topowright @BobBooth - Ready for Checklist verification. https://github.com/ArcGIS/solutions-defense-test-catalog/pull/136

dfoll commented 6 years ago

did checklist verification, but moving this to impeded, as Table To Ellipse is not working correctly. 317

topowright-zz commented 6 years ago

Looking at the table to ellipse tool again the issue that we are experiencing is not present in ArcGIS Pro 2.2, ArcMap 10.6 but is an issue in 10.6.1. I feel like the tool should have validation done before output is created that all fields are present. I am going to add that to the tool.

csmoore commented 6 years ago

Installing ArcGIS 10.6.1 (9269) to investigate the issue reported directly above on this version

csmoore commented 6 years ago

TableToEllipse seemed to run fine for me in 10.6.1.9269 (Distributor Pre Release) - so I don't seem to be able to repro @topowright 's issue mentioned here, but let me know if I missed something

image

The only remaining issue that I see with this toolset is a number of failures(exceptions) in the unit test driver with conversion_tests.TableToTwoPointLineTestCase on Pro 2.2.12776 (Beta2 released on6.4.18 ) so looking into this issue.

INFO: 2018-06-08 06:49:21,958 ------------ Begin Tests ------------------
INFO: 2018-06-08 06:49:22,052 Platform: Windows-10-10.0.14393-SP0
INFO: 2018-06-08 06:49:22,052 Host Name: Dev0001346
INFO: 2018-06-08 06:49:22,052 Python Version 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]
INFO: 2018-06-08 06:49:22,052 Product: ArcGISPro, Version: 2.2, Installed on: 6/8/2018, Build: 12776.
INFO: 2018-06-08 06:49:22,052 -------------------------------------------
...
RESULTS =================================================
Number of tests run: 92
Number succeeded: 84
Number of errors: 5
Number of failures: 3
Number of tests skipped: 0
Percent passing: 91.3
=========================================================

ERROR: 2018-06-08 07:28:40,456 ERRORS ==================================================
ERROR: 2018-06-08 07:28:40,456 test_table_to_twopointline_GARS (conversion_tests.TableToTwoPointLineTestCase.TableToTwoPointLineTestCase)
ERROR: 2018-06-08 07:28:40,456 Traceback (most recent call last):
ERROR: 2018-06-08 07:28:40,472   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\utils\test\conversion_tests\TableToTwoPointLineTestCase.py", line 160, in test_table_to_twopointline_GARS
ERROR: 2018-06-08 07:28:40,472     "GARS", "GARS_2", None, self.outputLines, "GEODESIC")
ERROR: 2018-06-08 07:28:40,472   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 92, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,472   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 89, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,472   File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 496, in <lambda>
ERROR: 2018-06-08 07:28:40,472     return lambda *args: val(*gp_fixargs(args, True))
ERROR: 2018-06-08 07:28:40,472 arcgisscripting.ExecuteError: Start Time: Friday, June 8, 2018 6:51:11 AM
ERROR: 2018-06-08 07:28:40,472 Failed to execute. Parameters are not valid.
ERROR: 2018-06-08 07:28:40,472 ERROR 000012: JoinID already exists
ERROR: 2018-06-08 07:28:40,472 Failed to execute (AddField).
ERROR: 2018-06-08 07:28:40,472 Failed at Friday, June 8, 2018 6:51:11 AM (Elapsed Time: 0.04 seconds)
ERROR: 2018-06-08 07:28:40,472 Failed to execute (TableTo2PointLine).
ERROR: 2018-06-08 07:28:40,472 test_table_to_twopointline_GEOREF (conversion_tests.TableToTwoPointLineTestCase.TableToTwoPointLineTestCase)
ERROR: 2018-06-08 07:28:40,472 Traceback (most recent call last):
ERROR: 2018-06-08 07:28:40,472   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\utils\test\conversion_tests\TableToTwoPointLineTestCase.py", line 219, in test_table_to_twopointline_GEOREF
ERROR: 2018-06-08 07:28:40,472     "GEOREF", "GEOREF_2", None, self.outputLines, "GEODESIC")
ERROR: 2018-06-08 07:28:40,472   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 92, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,472   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 89, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,472   File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 496, in <lambda>
ERROR: 2018-06-08 07:28:40,472     return lambda *args: val(*gp_fixargs(args, True))
ERROR: 2018-06-08 07:28:40,487 arcgisscripting.ExecuteError: Start Time: Friday, June 8, 2018 6:51:59 AM
ERROR: 2018-06-08 07:28:40,487 Failed to execute. Parameters are not valid.
ERROR: 2018-06-08 07:28:40,487 ERROR 000012: JoinID already exists
ERROR: 2018-06-08 07:28:40,487 Failed to execute (AddField).
ERROR: 2018-06-08 07:28:40,487 Failed at Friday, June 8, 2018 6:51:59 AM (Elapsed Time: 0.04 seconds)
ERROR: 2018-06-08 07:28:40,487 Failed to execute (TableTo2PointLine).
ERROR: 2018-06-08 07:28:40,487 test_table_to_twopointline_MGRS (conversion_tests.TableToTwoPointLineTestCase.TableToTwoPointLineTestCase)
ERROR: 2018-06-08 07:28:40,487 Traceback (most recent call last):
ERROR: 2018-06-08 07:28:40,487   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\utils\test\conversion_tests\TableToTwoPointLineTestCase.py", line 199, in test_table_to_twopointline_MGRS
ERROR: 2018-06-08 07:28:40,487     "MGRS", "MGRS_2", None, self.outputLines, "GEODESIC")
ERROR: 2018-06-08 07:28:40,487   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 92, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,487   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 89, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,487   File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 496, in <lambda>
ERROR: 2018-06-08 07:28:40,487     return lambda *args: val(*gp_fixargs(args, True))
ERROR: 2018-06-08 07:28:40,487 arcgisscripting.ExecuteError: Start Time: Friday, June 8, 2018 6:52:33 AM
ERROR: 2018-06-08 07:28:40,487 Failed to execute. Parameters are not valid.
ERROR: 2018-06-08 07:28:40,503 ERROR 000012: JoinID already exists
ERROR: 2018-06-08 07:28:40,503 Failed to execute (AddField).
ERROR: 2018-06-08 07:28:40,503 Failed at Friday, June 8, 2018 6:52:33 AM (Elapsed Time: 0.04 seconds)
ERROR: 2018-06-08 07:28:40,503 Failed to execute (TableTo2PointLine).
ERROR: 2018-06-08 07:28:40,503 test_table_to_twopointline_USNG (conversion_tests.TableToTwoPointLineTestCase.TableToTwoPointLineTestCase)
ERROR: 2018-06-08 07:28:40,503 Traceback (most recent call last):
ERROR: 2018-06-08 07:28:40,503   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\utils\test\conversion_tests\TableToTwoPointLineTestCase.py", line 179, in test_table_to_twopointline_USNG
ERROR: 2018-06-08 07:28:40,503     None, self.outputLines, "GEODESIC")
ERROR: 2018-06-08 07:28:40,503   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 92, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,503   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 89, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,503   File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 496, in <lambda>
ERROR: 2018-06-08 07:28:40,503     return lambda *args: val(*gp_fixargs(args, True))
ERROR: 2018-06-08 07:28:40,503 arcgisscripting.ExecuteError: Start Time: Friday, June 8, 2018 6:53:10 AM
ERROR: 2018-06-08 07:28:40,503 Failed to execute. Parameters are not valid.
ERROR: 2018-06-08 07:28:40,503 ERROR 000012: JoinID already exists
ERROR: 2018-06-08 07:28:40,503 Failed to execute (AddField).
ERROR: 2018-06-08 07:28:40,503 Failed at Friday, June 8, 2018 6:53:10 AM (Elapsed Time: 0.06 seconds)
ERROR: 2018-06-08 07:28:40,503 Failed to execute (TableTo2PointLine).
ERROR: 2018-06-08 07:28:40,503 test_table_to_twopointline_UTM_BANDS (conversion_tests.TableToTwoPointLineTestCase.TableToTwoPointLineTestCase)
ERROR: 2018-06-08 07:28:40,518 Traceback (most recent call last):
ERROR: 2018-06-08 07:28:40,518   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\utils\test\conversion_tests\TableToTwoPointLineTestCase.py", line 239, in test_table_to_twopointline_UTM_BANDS
ERROR: 2018-06-08 07:28:40,518     "UTM_BANDS", "UTM_2", None, self.outputLines, "GEODESIC")
ERROR: 2018-06-08 07:28:40,518   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 92, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,518   File "C:\Users\chri0020\Documents\Github\military-tools-geoprocessing-toolbox\toolboxes\Military Tools.pyt", line 89, in TableTo2PointLine
ERROR: 2018-06-08 07:28:40,518   File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 496, in <lambda>
ERROR: 2018-06-08 07:28:40,518     return lambda *args: val(*gp_fixargs(args, True))
ERROR: 2018-06-08 07:28:40,518 arcgisscripting.ExecuteError: Start Time: Friday, June 8, 2018 6:53:47 AM
ERROR: 2018-06-08 07:28:40,518 Failed to execute. Parameters are not valid.
ERROR: 2018-06-08 07:28:40,518 ERROR 000012: JoinID already exists
ERROR: 2018-06-08 07:28:40,518 Failed to execute (AddField).
ERROR: 2018-06-08 07:28:40,518 Failed at Friday, June 8, 2018 6:53:47 AM (Elapsed Time: 0.04 seconds)
ERROR: 2018-06-08 07:28:40,534 Failed to execute (TableTo2PointLine).
ERROR: 2018-06-08 07:28:40,534 FAILURES ================================================

UPDATE: I can reproduce this TableToTwoPointLineTestCase error in ArcMap 10.3.1-10.6.1 also - I would guess this is a new error introduced by changing this field to TEXT in #319

image

csmoore commented 6 years ago

Running the conversion unit tests should work on all versions now (Pro 2.1-2.2, ArcMap 10.3.1-10.6.1) with PR #333

INFO: 2018-06-08 12:17:47,445 ------------ Begin Tests ------------------
INFO: 2018-06-08 12:17:47,474 Platform: Windows-10-10.0.14393-SP0
INFO: 2018-06-08 12:17:47,475 Host Name: Dev0001346
INFO: 2018-06-08 12:17:47,475 Python Version 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]
INFO: 2018-06-08 12:17:47,476 Product: ArcGISPro, Version: 2.2, Installed on: 6/8/2018, Build: 12776.
INFO: 2018-06-08 12:17:47,477 -------------------------------------------
...
RESULTS =================================================
Number of tests run: 39
Number succeeded: 39
Number of errors: 0
Number of failures: 0
Number of tests skipped: 0
Percent passing: 100.0
=========================================================
topowright-zz commented 6 years ago

Closing this issue with the idea that we will not be writing any further unit tests at this point. We will open individual issues for tests that will need to be written for each unit test.

topowright-zz commented 6 years ago

@lfunkhouser I am closing this issue.

lfunkhouser commented 6 years ago

@topowright This was not a "won't fix". This issue was not just about creating unit tests. The acceptance criteria has changed. The conversion tools are in a .pyt and will no longer be released in the .tbx.