search

Esri / arcgis-python-api

Documentation and samples for ArcGIS API for Python
https://developers.arcgis.com/python/
Apache License 2.0
1.81k stars 1.08k forks source link

Publish_Layers not returning a StatusJob object when future set to True #1483

Closed FeralCatColonist closed 1 year ago

FeralCatColonist commented 1 year ago

Describe the bug

PortalDatastore.publish_layers() returns a boolean instead of a StatusJob object when the parameter future is set to true.

To Reproduce

Steps to reproduce the behavior:

import arcgis

def find_entry_dict_list(key, value, dict_list):
    for entry in dict_list:
        if entry[key] == value:
            return entry

gis = arcgis.gis.GIS(r"https://portal.domain.com/portal", "SOME_USER", "SOME_PASS")
gis_servers = gis.admin.federation.servers
gis_user = arcgis.gis.User(gis, "SOME_USER")
portal_datastore = gis.datastore

hosting_server = find_entry_dict_list("url", "https://arcgisserver.domain.com/server", gis_servers["servers"])
folder_ago_collaboration = find_entry_dict_list("title", "AGO_Collaboration", gis_user.folders)
ds_ago_collaboration = gis.content.search(query="title:AGO_Collaboration", item_type="Data Store")[0]

service_template_with_sync = {
    "serviceName": None,
    "type": "MapServer",
    "capabilities": "Map,Query,Data",
    "extensions": [{"typeName": "FeatureServer",
                    "capabilities": "Query,Sync",
                    "enabled": "true",
                    "properties": {
    "enableZDefaults": "false",
    "allowUpdateWithoutMValues": "false",
    "allowGeometryUpdates": "true",
    "editorTrackingTimeInUTC": "true",
    "allowOthersToQuery": "true",
    "includeTopologyLayer": "false",
    "syncBranchVersionCreationRule": "versionPerDownloadedMap",
    "enableOwnershipBasedAccessControl": "false",
    "onlyAllowTrueCurveUpdatesByTrueCurveClients": "false",
    "editorTrackingRespectsDayLightSavingTime": "false",
    "syncEnabled": "true",
    "syncVersionCreationRule": "versionPerDownloadedMap",
    "allowOthersToUpdate": "false",
    "zDefaultValue": "0",
    "allowTrueCurvesUpdates": "false",
    "xssPreventionEnabled": "true",
    "allowOthersToDelete": "false",
    "editorTrackingTimeZoneID": "UTC",
    "realm": ""
                    }}]
}

service_template = {"serviceName": None,
                    "type": "MapServer",
                    "capabilities":"Map,Query",
                    "extensions": [{"typeName": "FeatureServer",
                                    "capabilities":"Query,Create,Update,Delete",
                                    "enabled": "true",
                                    "properties": {"maxRecordCount": 3500}}]}

bulk_publish_job = portal_datastore.publish_layers(item= ds_ago_collaboration,
                                                   srv_config= service_template,
                                                   server_id= hosting_server["id"],
                                                   folder= folder_ago_collaboration["id"],
                                                   server_folder= "AGO_Collaboration",
                                                   sync_metadata= True,
                                                   use_config= True,
                                                   future= True)

bulk_publish_job returns as bool instead of StatusJob object. This occurs in both instances of the service_template_with_sync and the service_template from the API sample.

https://developers.arcgis.com/python/api-reference/arcgis.gis.toc.html#arcgis.gis._impl._datastores.PortalDataStore.publish_layers

Screenshots

image

image

Expected behavior

publish_layers() should return a StatusJob object when parameter future is set to true; otherwise an error should raise to explain the failure.

Platform (please complete the following information):

Additional context

The goal of this process is to automate a bulk publish to enable a Portal to ArcGIS Online collaboration with a full copy of the feature layers, which requires sync to be enabled.

The provided code was based on the API sample:

image
nanaeaubry commented 1 year ago

Thanks for bringing this to our attention, we will take a look

nanaeaubry commented 1 year ago

We fixed this for the next release, the parameter was there but never used in the method so we removed it because it cannot be implemented at this time

FeralCatColonist commented 1 year ago

Hi @nanaeaubry, When I ran the publish_layers() without the Future parameter I was still unable to get a successful run-- --the function just returned True.

nanaeaubry commented 1 year ago

@FeralCatColonist

In the method we wait for the response to say it is completed and then we return True once that is done. So if you are getting True back it means the post call to the rest endpoint was successful.

If you try doing it through the rest endpoint in the browser do you get the same results?

Here is the rest API doc for it: https://developers.arcgis.com/rest/users-groups-and-items/publish-layers.htm

FeralCatColonist commented 1 year ago

I'll have to re-test and get back to you on that. In the interim, is there any additional documentation on the templateSvcConfig parameter? The REST API documentation in your link also does not expound on what are the minimum requirements for that.

nanaeaubry commented 1 year ago

@FeralCatColonist I didn't forget about this but I am just waiting on a response. Will update once I have one

nanaeaubry commented 1 year ago

@FeralCatColonist

Here is the response I got:

In the doc those should cover all the key examples--I believe you can just edit the maxRecordCount value, and then the capabilities for the map and feature services. When you bulk publish from the home app, it uses the following template service config json:  

{
  "serviceName": null,
  "type": "MapServer",
  "capabilities": "Map,Query,Data",
  "extensions": [
    {
      "typeName": "FeatureServer",
      "capabilities": "Query",
      "enabled": "true",
      "properties": {"maxRecordCount": "4000"}
    }
  ]
} 

At 11.1, it can also contain date-time information: 

"properties": {
  "datesInUnknownTimeZone": false,
  "dateFieldsTimezoneID": "Pacific Standard Time",
  "dateFieldsRespectsDayLightSavingTime": true,
  "preferredTimeZoneID": "Pacific Standard Time",
  "preferredTimeZoneRespectsDayLightSavingTime": true
},
FeralCatColonist commented 1 year ago

Based on that, I'm wondering, if you omit certain properties, will those be filled out according to some default parameters?

I'd really like to understand what is the minimum required to enable a service with sync capability.

service_template_with_sync = {
    "serviceName": None,
    "type": "MapServer",
    "capabilities": "Map,Query,Data",
    "extensions": [{"typeName": "FeatureServer",
                    "capabilities": "Query,Sync",
                    "enabled": "true",
                    "properties": {
    "enableZDefaults": "false",
    "allowUpdateWithoutMValues": "false",
    "allowGeometryUpdates": "true",
    "editorTrackingTimeInUTC": "true",
    "allowOthersToQuery": "true",
    "includeTopologyLayer": "false",
    "syncBranchVersionCreationRule": "versionPerDownloadedMap",
    "enableOwnershipBasedAccessControl": "false",
    "onlyAllowTrueCurveUpdatesByTrueCurveClients": "false",
    "editorTrackingRespectsDayLightSavingTime": "false",
    "syncEnabled": "true",
    "syncVersionCreationRule": "versionPerDownloadedMap",
    "allowOthersToUpdate": "false",
    "zDefaultValue": "0",
    "allowTrueCurvesUpdates": "false",
    "xssPreventionEnabled": "true",
    "allowOthersToDelete": "false",
    "editorTrackingTimeZoneID": "UTC",
    "realm": ""
                    }}]
}

service_template = {"serviceName": None,
                    "type": "MapServer",
                    "capabilities":"Map,Query",
                    "extensions": [{"typeName": "FeatureServer",
                                    "capabilities":"Query,Create,Update,Delete",
                                    "enabled": "true",
                                    "properties": {"maxRecordCount": 3500}}]}
nanaeaubry commented 1 year ago

@FeralCatColonist

I asked for further clarification and received this from the team in charge of the doc for it: As far as I am aware, Sync (and to that extent, Editing as well) cannot be enabled on feature services with the initial bulk publish. It can be enabled after, however, in ArcGIS Server Manager. That's discussed in the front-end user help, but I see where that same information and clarification is missing in the API.

They went on to say they will make sure to update the doc to include these limitations.

Hope this is able to answer your questions!

FeralCatColonist commented 1 year ago

@nanaeaubry just to make sure I'm not misreading this, the team is saying there is not currently a programmatic way to set service capabilities such as sync or edit?

If not, is that capability on a development road map?

nanaeaubry commented 1 year ago

@FeralCatColonist Yes you are correct. You cannot currently do this on the bulk publish (publishLayers) it has to be done afterwards.

I don't know about development for this. The best thing to do is put in a feature request with support for this so the team has a user story they can address. Sorry for the inconvenience on that

FeralCatColonist commented 1 year ago

@nanaeaubry outside of the bulk publish, are those capabilities able to be set utilizing the API for Python on an individual basis? I wouldn't mind looping through services and setting those if I'm able. I've been button clicking through ArcGIS Server Manager and it is a beat down.

nanaeaubry commented 1 year ago

So you can set them afterwards with the python API This is a good guide to show you how to update the service definition: https://developers.arcgis.com/python/guide/updating-feature-layer-properties/