crackernutter / EsriRESTScraper

A Python class that scrapes ESRI Rest Endpoints and exports data to a geodatabase
MIT License
48 stars 20 forks source link

Sometimes the Spatial Reference is published as a wkt #3

Closed mmdolbow closed 8 years ago

mmdolbow commented 8 years ago

This is probably fairly rare, but I've come across it at least once: instead of using a wkid, the spatial reference of a layer is described as thus in a REST endpoint JSON: spatialReference: { wkt: "PROJCS.... }

Thus breaking the code. I looked around for a wkt to wkid conversion with arcpy, but it looks like one would have to obtain the string that describes the spatial reference and then create it from that string (which isn't the entire string described in the JSON).

crackernutter commented 8 years ago

Thanks! Can you send me an endpoint that specifies spatial reference in wkt instead of wkid and I'll look into it.
Thanks!

mmdolbow commented 8 years ago

Yes, here's one: https://sampleserver1.arcgisonline.com/ArcGIS/rest/services/TaxParcel/ConsumerFocusedPublicAccessMap/MapServer

crackernutter commented 8 years ago

Okay, so I've done some research, and I think this issue may have more to due with system changes that Esri made to server between the 10.0 and 10.1 releases (the service you sent me is from a 10.0 server).

Here is what I've found.

Creating a SpatialReference object in arcpy with a WKID of 103079, and then exporting that to its wkt string results is:

PROJCS['NAD_1983_2011_StatePlane_Michigan_South_FIPS_2113_Ft_Intl',GEOGCS['GCS_NAD_1983_2011',DATUM['D_NAD_1983_2011',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Lambert_Conformal_Conic'],PARAMETER['False_Easting',13123359.58005249],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',-84.36666666666666],PARAMETER['Standard_Parallel_1',42.1],PARAMETER['Standard_Parallel_2',43.66666666666666],PARAMETER['Latitude_Of_Origin',41.5],UNIT['Foot',0.3048]];-107138600 -97193800 3048;-100000 10000;-100000 10000;3.28083989501312E-03;0.001;0.001;IsHighPrecision.

The wkt string in the web service you provided is:

PROJCS["NAD_1983_StatePlane_Michigan_South_FIPS_2113_IntlFeet",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Lambert_Conformal_Conic"],PARAMETER["False_Easting",13123359.58005249],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-84.36666666666666],PARAMETER["Standard_Parallel_1",42.1],PARAMETER["Standard_Parallel_2",43.66666666666666],PARAMETER["Latitude_Of_Origin",41.5],UNIT["Foot",0.3048]]

They're basically the same with a couple cosmetic differences (_FtIntl vs IntlFeet for instance). So my guess is the wkt's for spatial references changed between 10.0 and 10.1. Granted, the RestScraper doesn't account for services published with a wkt, and although it would be easy to include that, it would still not work on your example service because arcpy doesn't recognize it. I could try to reverse engineer any string differences between versions, but frankly I don't have the time.

I've encountered another problem with pre 10.1 web services - they don't have a returnCountOnly query parameter, which also breaks my code.

In light of this, I'm just thinking about including that this utility is only compatible with 10.1 services and above.

Hopefully this makes sense, and sorry I couldn't be of more help!

Jay

mmdolbow commented 8 years ago

I think it's a great idea to focus on post 10.1 services - you're right on that returnCount param, that's critical to executing my usage of the code as well. But I should have dug deeper and given you a post 10.1 endpoint. Here's one: http://dswcapps.dcr.virginia.gov/arcgis/rest/services/bmp_query/MapServer

If you want to find more, just do a Google search on "Spatial Reference: PROJCS" "arcgis/rest/services"

And, no worries if you don't have time to fix! :-) Maybe I will at some point!

crackernutter commented 8 years ago

Interesting. I wonder why the server would publish using WKT vs WKID. According to this it's because the publisher created a custom spatial reference.

Curious what would happen if you just tried creating a spatial reference based on the wkt string and try to add geometries to the feature class....