search

Esri / arcgis-python-api

Documentation and samples for ArcGIS API for Python
https://developers.arcgis.com/python/
Apache License 2.0
1.87k stars 1.1k forks source link

datastore.list() performance issue #1179

Closed azinsharaf closed 2 years ago

azinsharaf commented 2 years ago

Describe the bug datastores.list() is very slow. It takes about 90 seconds to generate the list for a GIS server with 60 datastore items.

To Reproduce Steps to reproduce the behavior:

gis_servers = gis.admin.servers.list()
    for gis_server in gis_servers:
        if "geoportal" in gis_server.url.lower():
            data_store_list = gis_server.datastores.list()

Expected behavior faster result

Platform (please complete the following information):

achapkowski commented 2 years ago

Can you provide more about your configuration?

azinsharaf commented 2 years ago

I am using ArcGIS Enterprise 10.9.1 (base deployment). The hosted server has about 60 registered datastore items (most of them are enterprise geodatabases in SQL Server 2017). When I run gis_server.datastores.list() it takes at least 1 minute to generate a list.

achapkowski commented 2 years ago

ok, we'll take a look.

achapkowski commented 2 years ago

How is your internal network speed and the communication between the egdb and the hosting enterprise? It sounds like there might be some lag. Is the network traffic prioritized?

azinsharaf commented 2 years ago

Hi @achapkowski , I think the internal network has enough bandwidth. The feature services render quickly (less than 2sec) from enterprise geodatabase. Also when I "Validate" the data stores in ArcGIS Manager interface, it is fast.

achapkowski commented 2 years ago

If it is 1.5 seconds to validate one data store and you have 60. That seems to fall inline with the 90 seconds reported.

I did add some logic in 2.0.1 to lazy load some properties, which reduced the class creation by ~10-15% the rest of the time is spent on querying the server for the registered data store items.

azinsharaf commented 2 years ago

Got it. Does it mean the taken time for list.dataset is similar to validate?