Esri / data-assistant

ArcGIS Pro Add-in that assists in emergency management, local government and state government data aggregation workflows.
Apache License 2.0
22 stars 8 forks source link

New mapping file fails to complete #58

Closed jessnoo closed 8 years ago

jessnoo commented 8 years ago

With the lastest development cut on Pro 1.3, the new mapping file tool completes successfully but does not create a file. It reports that the target layer isn't a feature service layer when it is.

steps to repro:

  1. Publish district dataset shared in issue #53 by Lindsay T as a hosted feature service in ArcGIS Online (attached below for reference)
  2. Create a new map in pro with the source FC and published feature service as layers
  3. Click "New File"
  4. Set source as the local fgdb district feature class
  5. Set target as the published feature service
  6. Run tool -- receive message: image PublicWorksUpdates.gdb.zip

Also reproed with the parcel layer shipped with Community Parcels image

@SteveGrise what do you think?

FYI @NikkiGolding @ChrisBuscaglia

jessnoo commented 8 years ago

point of clarification -- i'm using the released 1.3 version. (NOT a daily or special build)

SteveGrise commented 8 years ago

I have been working on 1.2, will need to do some testing to prepare for 1.3 release.

On Thu, Aug 25, 2016 at 2:48 PM, Jess Neuner notifications@github.com wrote:

point of clarification -- i'm using the released 1.3 version. (NOT a daily or special build)

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/Esri/data-assistant/issues/58#issuecomment-242498112, or mute the thread https://github.com/notifications/unsubscribe-auth/AEh3BBZQPDbkYPloU859zJfNbgo9jQCmks5qjeOFgaJpZM4JtYOf .

ChrisBuscaglia commented 8 years ago

@jessnoo Does this work in ArcGISPro 1.2? There was this issue that was fixed...#51 Also - did the XML actually save the info and run successfully?

jessnoo commented 8 years ago

@ChrisBuscaglia i didn't test in 1.2, i can circle back on that if we feel like it's relevant?

no -- the tool did not create any file -- there are kind of two issues here:

  1. the tool says it's not a feature service layer when it is
  2. the tool says it finished successfully when it didn't
ChrisBuscaglia commented 8 years ago

@jessnoo I'm trying to determine if the ArcGIS Pro 1.2 exhibits the same behavior, that would be really bad since it's the version that the tool is supported on.

jessnoo commented 8 years ago

@ChrisBuscaglia i think we are ok with the released tools. with the september release, i believe we are planning to certify all the Pro solutions on 1.3? i will find some time to install 1.2 and do another test.

jessnoo commented 8 years ago

@ChrisBuscaglia @SteveGrise

Retested with 1.2 and the New File tool ran successfully (iow passed the test) with the same workflow described above.

image

looks like there is a bug at 1.3 with the August version of the New File tool.

@NikkiGolding FYI

SteveGrise commented 8 years ago

I believe this 1.3 issue is similar to what someone had found at the UC. It looks like something changed in the GP Parameters in 1.3, hopefully it addresses the workaround we did in 1.2 to catch feature service vs feature layer selections.

On Thu, Aug 25, 2016 at 7:54 PM, Jess Neuner notifications@github.com wrote:

@ChrisBuscaglia https://github.com/ChrisBuscaglia @SteveGrise https://github.com/SteveGrise

Retested with 1.2 and the New File tool ran successfully (iow passed the test) with the same workflow described above.

[image: image] https://cloud.githubusercontent.com/assets/6181441/17989652/ed700b56-6ae3-11e6-8646-9607e9cb5584.png

looks like there is a bug at 1.3 with the August version of the New File tool.

@NikkiGolding https://github.com/NikkiGolding FYI

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Esri/data-assistant/issues/58#issuecomment-242582304, or mute the thread https://github.com/notifications/unsubscribe-auth/AEh3BK1k0ww6NP_YZ4xaM6NpvfVHb3_kks5qjiskgaJpZM4JtYOf .

ChrisBuscaglia commented 8 years ago

@SteveGrise

I can reproduce this with the 1.2 released addin with ArcGIS Pro 1.3.

I did however get around the problem by dragging and dropping the feature service directly into the tool from my portal (appending the layerid).

8-26-2016 10-53-13 am

Since every customer with 1.2 has been prompted to install ArcGIS Pro 1.3, they will hit this issue. We'll need to get it addressed for our Sept. release.

Can you find some time to take a look and estimate how long a fix may take? I've looked into the tool to see where I could get around the feature layer check, but it's not a sound fix.

@NikkiGolding @jessnoo

Chris

SteveGrise commented 8 years ago

@ChrisBuscaglia sure, I can take a look. I'm sure there's a fix for it, shouldn't be too hard.

SteveGrise commented 8 years ago

Seems to me like there is a significant change in service urls in ArcGIS Pro 1.3(.1). I'm pretty sure in earlier versions that catalogPath and path properties for services would return the /0 (/id) style urls. Now I see url strings like '/L0PARCELS'.

@ChrisBuscaglia I'd like to verify that this behaviour is now 'normal' in Pro. It's a big change.

I've worked around it and I believe the issue is resolved in the August2016 branch.

service string

NikkiGolding commented 8 years ago

@chris-fox can you talk with the Pro team and get some info regarding catalogPath and path properties moving forward?

chris-fox commented 8 years ago

@NikkiGolding, I spoke with the GP team and this is not expected behavior, they were actually already aware of it and had logged a bug, CR340485. The expected behavior is it should return the url to the layer or table within the feature service.

SteveGrise commented 8 years ago

Ok, I worked around it in the code - if the layer id startswith 'L' then it will strip off the L and drop the feature service name that is appended after the numeric layerid. If just the numeric value is passed then the code does nothing.

I did notice that arcpy.Describe works better in 1.3, looks like some of the complexity in figuring out layer properties (path) is no longer needed.

On Thu, Sep 1, 2016 at 9:08 AM, Chris Fox notifications@github.com wrote:

@NikkiGolding https://github.com/NikkiGolding, I spoke with the GP team and this is not expected behavior, they were actually already aware of it and had logged a bug, CR340485. The expected behavior is it should return the url to the layer or table within the feature service.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Esri/data-assistant/issues/58#issuecomment-244073129, or mute the thread https://github.com/notifications/unsubscribe-auth/AEh3BDIfcRGmAcuvNpp93cbaWxuwmOtpks5qls5egaJpZM4JtYOf .

ChrisBuscaglia commented 8 years ago

@NikkiGolding Just getting to this, has anyone tested this?

NikkiGolding commented 8 years ago

@ChrisBuscaglia not yet. @jessnoo did not get to it before she left on vacation. @lindsayking will work on this today so we can get moving on getting it signed.

lindsayking commented 8 years ago

@ChrisBuscaglia @NikkiGolding When testing with the latest in the August branch, I am still encountering an issue using Jess's repro steps in the original comment above. 1) Improvement: the tool no longer reports that it completes successfully when it doesn't (it no longer reports success when it is really a failure) 2) However, the tool fails, reporting that the PublicWorksDistrict does not appear to be a feature service layer. It also says that PublicWorksDistricts does not have create and delete privileges, but it does, according to its endpoint.

fails

Note, the tool DOES complete successfully when - instead of dragging the feature service layer into the input from my contents pane - I browse to the feature service in my content and input it that way: passes

NikkiGolding commented 8 years ago

@SteveGrise Can you take a look?

SteveGrise commented 8 years ago

In my testing the layer was prefixed with an 'L', like '/L0PublicWorksDistrict'. In this example it looks like just '/0PublicWorksDistrict'. I'm just guessing at the logic here since I am working around a bug in Pro (Chris Fox referenced it but I don't think I have access to the bug to read about it), but maybe I should change the logic so that I basically just extract the number from this string and drop any letters? @lindsayking @NikkiGolding @chris-fox

SteveGrise commented 8 years ago

Made a change to just take the first integer in the last part of the layer url. Committed to the August 2016 branch.

lindsayking commented 8 years ago

@SteveGrise Thanks. I'll re-test now.

lindsayking commented 8 years ago

@SteveGrise @NikkiGolding @ChrisBuscaglia

The latest change looks good. Using the same workflow as above, I am now seeing the tool complete successfully. fixpasses

My only concern is that it now looks like it checks to service/REST capabilities twice. Is that an issue, or are we good to proceed with what we have?

SteveGrise commented 8 years ago

@lindsayking I think you used the same source and target? If you use a file gdb/feature service combo it should check each one separately.

lindsayking commented 8 years ago

@SteveGrise - I used a file gdb feature class as the source dataset and the feature service layer as the target. In this case, I dragged them into the tool from my contents pane, and they had the same name. This is the scenario in which I see the checks to the service twice.

When I change the name of the file gdb layer and re-run the tool, however, I don't see the checkts reported twice. I'm using the same inputs as in the first scenario, just with the fgdb layer renamed.

SteveGrise commented 8 years ago

@lindsayking the issue is that from the GP script I just get the layer name and then it's just picking up the first layer with that name both times. It's actually not going to create the file the way you want.

I'm not sure if I can figure out from the GP tool parameter which of the layers with the same name to use, this all happens before the script runs. For example, in Pro GP tool I see PublicWorksDistrict:1 and :2 in the dialog after dragging, but in the script I only receive: PublicWorksDistrict without any of the sequence info.

This is a different issue than the one we saw before, I'm not sure how to work around it.