Bolton-and-Menk-GIS / restapi

Python API designed to work externally with ArcGIS REST Services to query and extract data, and view service properties. Uses arcpy for some functions if available, otherwise uses open source alternatives to interact with the ArcGIS REST API. Also includes a subpackage for administering ArcGIS Server Sites.
GNU General Public License v2.0
93 stars 31 forks source link

Failing on generate SHP #15

Closed mfandre closed 4 years ago

mfandre commented 6 years ago

My code:

# -*- coding: utf-8 -*-
import restapi

rest_url = "http://app.anp.gov.br/arcgis/rest/services"
ags = restapi.ArcServer(rest_url)
print ags

ANP_Public_Confidential = ags.getService('ANP_Public_Confidential')

#listlayers
print ANP_Public_Confidential.list_layers()

count = 1
for lyrName in ANP_Public_Confidential.list_layers():
    print "generating:" + lyrName
    lyr = ANP_Public_Confidential.layer(lyrName)
    output = "c:\\temp\\a" + str(count) + ".shp"
    lyr.layer_to_fc(output)
    count = count + 1

The stack trace:


  File "C:\Anaconda\lib\site-packages\restapi\shapefile\shapefile.py", line 1083, in save
    self.saveShp(target)

  File "C:\Anaconda\lib\site-packages\restapi\shapefile\shapefile.py", line 1041, in saveShp
    self.__shpRecords()

  File "C:\Anaconda\lib\site-packages\restapi\shapefile\shapefile.py", line 833, in __shpRecords
    raise ShapefileException("Failed to write points for record %s. Expected floats." % recNum)

ShapefileException: Failed to write points for record 2. Expected floats.
eanderson4 commented 5 years ago

I would be interested in this as well.

I had issues with esriGeometryPolyline writing to the shapefile. As I have dug in, it seems as though the bounding box ending up having an array instead of a float.

Not sure if it has to do with versions, but I was using the latest version of this package and was running on python 2.7, and I don't have arcpy.

Edit: The issue seems to be coming from the following in the code there are quite a few statements like this that are being hit (ex. shapefile line 375-379)

            XMin = min(coords[0] for coords in shp.points)
            YMin = min(coords[1] for coords in shp.points)
            XMax = max(coords[0] for coords in shp.points)
            YMax = max(coords[1] for coords in shp.points)

It seems like it is expecting an array of coordinates, but printing my shp.points, it is actually an array of array of coordinates.

# example shp.points
[[[-9749783.0243, 4577845.705700003], [-9749808.3075, 4577861.058899999], [-9749807.1156, 4577947.265699998], [-9749914.3921, 4577949.7863000035], [-9749910.3216, 4578143.903800003], [-9749878.3064, 4578147.577399999]]]

So it is taking a minimum of a list of coordinates, versus a list of floats. The minimum of this is an array not a float which is what the error is saying.

Perhaps the input data is not being parsed properly?

Here is the mapserver + layer I'm working with.

mapserver = restapi.MapService(mapserver_url)
mapserver_url = "https://maps.eia.gov/arcgis/rest/services/USEnergyMappingSystem/20190205/MapServer/"

lines = mapserver.layer('Electric Transmission Line')
lines.layer_to_fc('tmp/transmission_line.shp',exceed_limit=True)
eanderson4 commented 5 years ago

Just to leave a quick follow up.

I ended up doing a small patch to get it to export. The particular layer I was interested in seemed to not have many multi line-strings. So when parsing the geometry, I just took the first piece of the line string as follows (in shapefile.py on line 280)

              elif PATHS in geometry:
                    if len(geometry[PATHS])==1:
                        self.json[PATHS] = geometry[PATHS][0]
                    else:
                        print('Error, not handling multipath')
                        #raise IOError('Not handling Multipaths')

After this, I was finally able to write the layer to a shapefile and when loaded in QGIS it appears to be correct.

I do see a note on trying to fix multis above and it uses some sort of part system. The coords extend is commented out and I didn't test to see if that approach would work. For now this will do.

Thanks for your nice package! The alternative seems to be doing this manually through the Arc REST API plugin in QGIS which I don't know how to automate.

CalebM1987 commented 5 years ago

Sorry for the late response here, but I just updated this repo to use the newest version of the shapefile package (2.1), which is what handles writing the shapefiles to disk. Can you see if that resolves this issue?

philnagel commented 4 years ago

Haven't heard a response, I assume this was resolved. Feel free to re-open if the issue persists.