Get WFS data from ArcGIS Server

[michelmetran]

Hello,

I found the StElt project and I believe it can help me! I'm looking for ways to download, in GML format, files that are on a WFS server. However, they are on an ArcGIS server and not on a Geoserver.

• Friendly Interface of ArcGIS Server:
• URL to get GML file
• Example to GML file

Using the ows library I managed to obtain a lot of data, just a layer that is impossible to get, due to its size (about 1,200,000 features).

I'm studying StEtl, from example/basics/8_wfs... but I am not successful. My gml file gets the following message: b"<ows:ExceptionReport version='1.1.0' language='en' xmlns:ows='http://www.opengis.net/ows'><ows:Exception exceptionCode='NoApplicableCode'><ows:ExceptionText>Xml Parsing failed.\r\nReason:A semi colon character was expected.\r\n\r\nLinepos:31\r\nSource:srsName=EPSG:4674&amp;outputFormat=text/xml;%20subtype=gml/2.1.2&amp;typename=Banco_de_Areia_-_RMSP_RMBS&amp;request=GetFeature&amp;service=WFS&amp;version=1.1.0.</ows:ExceptionText></ows:Exception></ows:ExceptionReport>"

It's possible to get the data from these ArcGIS Server using StElt? Thanks!

[justb4]

So you made a Stetl config for a WFS Input similar to the example and ArcGIS Server responds with the above ExceptionReport? For determining what causes the failure we need to see your Stetl config.

Can you maybe clarify your aim a bit more: you e.g. say "I'm looking for ways to download, in GML format, files that are on a WFS server". Just downloading GML? Or also processing that GML into e.g. PostGIS?

WFS-es can be restrictive in the amount of Features one may download. In that case one may need to download blocks using a WFS Bounding Box Filter.

I am downloading the Layer IGC_BaseHidrografica_25000_WFS using wget on the commandline, but it takes forever...

[michelmetran]

Hello!, Thank you for your help!

As I am a StElt apprentice, I chose to export the result in a GML file. After that, I can try PostGIS! For now, I just changed the configuration file to save the file in GML in a directory "output", using a small layer to tests.

[etl]
chains = input_wfs|output_file

[input_wfs]
class = stetl.inputs.httpinput.HttpInput
url = http://geoportal.igc.sp.gov.br:6080/arcgis/services/WFS/IGC_BaseHidrografica_25000_WFS/MapServer/WFSServer
#url = http://geoportal.igc.sp.gov.br:6080/arcgis/services/WFS/IGC_BaseHidrografica_25000_WFS/MapServer/WFSServer?
#url = http://geoportal.igc.sp.gov.br:6080/arcgis/rest/services/WFS/IGC_BaseHidrografica_25000_WFS/MapServer
parameters = {
    'service' : 'WFS',
    'version' : '1.1.0',
    'request' : 'GetFeature',
    'srsName' : 'EPSG:4674',
    'outputFormat' : 'text/xml; subtype=gml/2.1.2',
    'typename' : 'Rocha_em_Água_-_RMSP_RMBS',
    }

[output_file]
class = stetl.outputs.fileoutput.FileOutput
file_path = output/gml_igc.gml

I have tried using the ows package and, with it, I was able to bypass the restriction in the amount os features defined in the ArcGIS Server! I downloaded all the complete GML files! Is just missing that huge one.

The second-largest file has 1 million fewer features (I found this out with queries that return only the ID, without the spatial component). I take some hours to download. The huge layer broke (and, of course, is the most important!!).

I noticed that the download restriction only occurs using the REST in URL, to obtain .geojson files. Using another link, without a friendly interface, it is possible to get the complete GML files! Below is the second-largest layer.

Thanks!

[justb4]

ESRI OGC services like WFS are sometimes not 100% standards-compliant. The error you get complains about a semi colon, ';'.

Try to leave out outputFormat from your Stetl config: 'outputFormat' : 'text/xml; subtype=gml/2.1.2',. By default the wfs should return GML as in URL example.

[justb4]

@michelmetran is your problem solved? Then we can close this issue.

[michelmetran]

Yeap! I'll close ok. Thanks for your help! The stETL helps me!