Bolton-and-Menk-GIS / restapi

Python API designed to work externally with ArcGIS REST Services to query and extract data, and view service properties. Uses arcpy for some functions if available, otherwise uses open source alternatives to interact with the ArcGIS REST API. Also includes a subpackage for administering ArcGIS Server Sites.
GNU General Public License v2.0
93 stars 31 forks source link

fields kwarg is ignored when using exceed_limit #55

Closed marceloandrioni closed 3 months ago

marceloandrioni commented 3 months ago

Hello, I think I found a problem when using the fields kwarg and exceed_limit in the same request. An example would be:

import restapi

rest_url = 'https://sampleserver6.arcgisonline.com/arcgis/rest/services'
ags = restapi.ArcServer(rest_url)
usa = ags.getService('USA') #/USA/MapServer -> restapi.common_types.MapService
cities = usa.layer('Cities') # or can use layer id: usa.layer(0)

where = "st = 'CA' and pop2000 > 100000"

# OK: getting all fields, same as fields='*'
featureSet = cities.query(where=where)
print(featureSet[0]["properties"])
# {
#   "areaname": "Anaheim",
#   "capital": "N",
#   "class": "city",
#   "objectid": 169,
#   "pop2000": 328014,
#   "st": "CA"
# }

# OK: getting some fields in a single query
featureSet = cities.query(where=where, fields=["areaname", "capital"])
print(featureSet[0]["properties"])
# {
#   "areaname": "Anaheim",
#   "capital": "N"
# }

# FAIL: getting some fields in a query_in_chunks. All fields are returned.
featureSet = cities.query(where=where, fields=["areaname", "capital"],
                          exceed_limit=True)
print(featureSet[0]["properties"])
# {
#   "areaname": "Anaheim",
#   "capital": "N",
#   "class": "city",
#   "objectid": 169,
#   "pop2000": 328014,
#   "st": "CA"
# }

Checking the code I think I found the problem in this line In this line the params dict was already parsed using the _validate_params and the fields kwargs became outFields. Then, when query_in_chunks is called with outFields the list of fields is ignored and all are returned. I believe the fix would be to replace

for i, result in enumerate(self.query_in_chunks(records=records, **params)):

with

for i, result in enumerate(self.query_in_chunks(records=records, where=where, fields=fields, chunk_size=chunk_size, **kwargs)):

Should I make a PR?

Thank you.

CalebM1987 commented 3 months ago

Interesting. That does appear to be a bug. We would definitely welcome the PR if that does indeed fix that. Nice catch!