Allow ArcGIS Rest Layers to be added dynamically

[geographika]

The AddWms form and button were integrated in cpsi-mapview as part of #250.

It is desirable to have similar functionality to add ArcGIS Server (and AGOL) REST layers in a similar manner - by connecting to a server to display the available layers and then to allow users to select them to add them to a map.

This functionality may have previously been considered for the BasiGX library. If an ArcGIS REST parser is required, it may make sense to add it to the code GeoExt library.

The connector may have to support access tokens. Some sample URLs can be provided.

[geographika]

Sample (open) ArcGIS servers with data for Ireland:

https://services6.arcgis.com/uWTLlTypaM5QTKd2/ArcGIS/rest/services https://services7.arcgis.com/e7ziY0LPPjwxz0bF/ArcGIS/rest/services https://gis.epa.ie/arcgis/rest/services/

[jansule]

Are there any requirements to support Internet Explorer? I am thinking about using the URL interface, which is not supported by IE.

[geographika]

Yes, unfortunately many users are still on IE11. We have added in polyfills to the project, so maybe the following would allow compatibility? https://www.npmjs.com/package/url-polyfill In a dependent project I use https://polyfill.io/v3/polyfill.min.js?features=fetch%2CrequestAnimationFrame%2CElement.prototype.classList%2CURL%2CTextDecoder%2CNumber.isInteger%2CArray.prototype.flat to generate a polyfill and then add this to app.json:

  "js": [
    // polyfills for IE11 - see https://github.com/openlayers/openlayers/blob/main/README.md#supported-browsers
    // generated from https://polyfill.io/v3/url-builder/
    // also includes https://developer.mozilla.org/fr/docs/Web/JavaScript/Reference/Objets_globaux/Array/flat
    // https://polyfill.io/v3/polyfill.min.js?features=fetch%2CrequestAnimationFrame%2CElement.prototype.classList%2CURL%2CTextDecoder%2CNumber.isInteger%2CArray.prototype.flat
    //
    {
      "path": "./lib/polyfill.min.js"
    },

[jansule]

Base functionality added to BasiGX in https://github.com/terrestris/BasiGX/pull/691

[jansule]

Follow up from https://github.com/compassinformatics/cpsi-mapview/pull/582#issuecomment-1202763019

As noted in https://github.com/terrestris/BasiGX/pull/691 FeatureServer type layers are not supported. I'd imagine this would need a new custom parser in GeoExt to handle this?

This depends on how we want to work with FeatureServer type layers. ArcGIS FeatureServer is a collection of layers (and tables - not interesting for us) that contain the raw data incl. geometries. With OpenLayers, we are already able to display a SINGLE layer using the EsriJSON format in combination with a VectorLayer. I created a fiddle for this: https://codesandbox.io/s/vector-esri-forked-sxstkv.

With OpenLayers, we are NOT able to work with the collection of layers out of the box. We could add a custom loader that gathers the data from all layers and adds them into a single layer. However, I am not sure, if this is a use-case for us.

With regards to the UI, we could check the included layers of a FeatureServer, if an ArcGIS Service contains one. These could then be displayed alongside the other Services/Layers.

This would have a negative impact on the performance, though, since we fire an additional HTTP request for every FeatureServer (not layer) included.

@geographika Would the described approach/workaround for adding single layers of a FeatureServer be sufficient for your use-cases?

Even with the parser the data would be loaded as unstyled vector data (which would be a good start). Is this something that could be handled using GeoStyler? I see there is an issue at https://github.com/geostyler/geostyler/issues/1204 which looks relevant?

When we are using the above mentioned approach, we will be creating OpenLayers Vector Layers, so we can create OpenLayers Styles with GeoStyler,

[geographika]

@jansule - sounds very promising, and thanks for the Fiddle demo. I'm not sure why the example I sent has multiple FeatureServers, with each one only containing a single layer. The approach of looping through each and creating a flat list would be perfect for the application use cases. One thing to consider is how to label the nested layers. In the example of:

https://services7.arcgis.com/e7ziY0LPPjwxz0bF/ArcGIS/rest/services/survey123_cc5f381e879d445ca4b07f804ec07a99/FeatureServer

A combination of FeatureServer plus layer name could be the most useful e.g. survey123_cc5f381e879d445ca4b07f804ec07a99/survey. Other suggestions welcome.

As you note in https://github.com/terrestris/BasiGX/pull/691, the following is not supported:

import single layers of a service

Would adding in single layers for the FeatureServer provide an approach for single layers of the MapServer and ImageServer types? It is hard to guess which is more useful for a user, and this could change from one server setup to another.

With OpenLayers, we are NOT able to work with the collection of layers out of the box. We could add a custom loader that gathers the data from all layers and adds them into a single layer. However, I am not sure, if this is a use-case for us.

No, this is not required. I think for vector layers (and possibly all layers - see above), users will probably be looking to add a very specific single dataset to the system to view alongside the inbuilt data.

When we are using the above mentioned approach, we will be creating OpenLayers Vector Layers, so we can create OpenLayers Styles with GeoStyler.

Excellent!

[geographika]

@jansule - sounds very promising, and thanks for the Fiddle demo. I'm not sure why the example I sent has multiple FeatureServers, with each one only containing a single layer. The approach of looping through each and creating a flat list would be perfect for the application use cases. One thing to consider is how to label the nested layers. In the example of:

https://services7.arcgis.com/e7ziY0LPPjwxz0bF/ArcGIS/rest/services/survey123_cc5f381e879d445ca4b07f804ec07a99/FeatureServer

A combination of FeatureServer plus layer name could be the most useful e.g. survey123_cc5f381e879d445ca4b07f804ec07a99/survey. Other suggestions welcome.

As you note in https://github.com/terrestris/BasiGX/pull/691, the following is not supported:

import single layers of a service

Would adding in single layers for the FeatureServer provide an approach for single layers of the MapServer and ImageServer types? It is hard to guess which is more useful for a user, and this could change from one server setup to another.

With OpenLayers, we are NOT able to work with the collection of layers out of the box. We could add a custom loader that gathers the data from all layers and adds them into a single layer. However, I am not sure, if this is a use-case for us.

No, this is not required. I think for vector layers (and possibly all layers - see above), users will probably be looking to add a very specific single dataset to the system to view alongside the inbuilt data.

When we are using the above mentioned approach, we will be creating OpenLayers Vector Layers, so we can create OpenLayers Styles with GeoStyler.

Excellent!

[jansule]

@jansule - sounds very promising, and thanks for the Fiddle demo. I'm not sure why the example I sent has multiple FeatureServers, with each one only containing a single layer. The approach of looping through each and creating a flat list would be perfect for the application use cases. One thing to consider is how to label the nested layers. In the example of:

https://services7.arcgis.com/e7ziY0LPPjwxz0bF/ArcGIS/rest/services/survey123_cc5f381e879d445ca4b07f804ec07a99/FeatureServer

A combination of FeatureServer plus layer name could be the most useful e.g. survey123_cc5f381e879d445ca4b07f804ec07a99/survey. Other suggestions welcome.

I would go for the same approach. Would it be helpful for users to be able to visually distinguish the different service types? So that users would directly know, if they are adding a MapServer/FeatureServer/ImageServer.

As you note in terrestris/BasiGX#691, the following is not supported:

import single layers of a service

Would adding in single layers for the FeatureServer provide an approach for single layers of the MapServer and ImageServer types? It is hard to guess which is more useful for a user, and this could change from one server setup to another.

It is possible to add single layers for MapServer and ImageServer as well. The two *Servers kind of represent in the Esri realm what a WMS represents within the OGC standards - already rendered map(tile)s. What we are currently importing is the complete map that consists of the different layers (e.g. buildings, roads, labels). If we want to also be able to import single layers, we mainly have to think about how this could be properly visualized in the UI without overwhelming users.

[geographika]

Ok let's go with that naming convention. It would be nice to distinguish between tile and vector layers - would it be by grouping of types or with an icon or similar? This is a nice-to-have so can be dropped until a future sprint if necessary.

It is possible to add single layers for MapServer and ImageServer as well. The two *Servers kind of represent in the Esri realm what a WMS represents within the OGC standards - already rendered map(tile)s. What we are currently importing is the complete map that consists of the different layers (e.g. buildings, roads, labels). If we want to also be able to import single layers, we mainly have to think about how this could be properly visualized in the UI without overwhelming users.

I see - it has been awhile since I worked with ArcGIS Server. It makes sense to leave them as collections at this stage, and only have single layers for FeatureServer services.

[jansule]

Support for ArcGIS REST FeatureServer (single layers only) implemented in https://github.com/terrestris/BasiGX/pull/696. A screencast was added to the PR. Additional changes in cpsi-mapview (except for upgrading BasiGX) are not needed.

[geographika]

Thanks @jansule - tested locally and it works very well. Looking forward to it being merged into BasiGX. Do you think you will also be looking at the GeoStyler integration as part of this development?

[jansule]

Do you think you will also be looking at the GeoStyler integration as part of this development?

Sure, I could implement this as part of cspi-mapview. The question here is, where do you want to enable styling? Will it be on the layer in the layer tree? And/or should there be some popup after importing? If so, we would have to consider how to handle styling when importing multiple layers at once.

[geographika]

It looks like styling is stored in the drawingInfo property e.g.

https://services7.arcgis.com/e7ziY0LPPjwxz0bF/ArcGIS/rest/services/Drainage_Data/FeatureServer/4

    {"renderer":{"type":"simple","symbol":{"type":"esriSLS","style":"esriSLSSolid","color":[112,168,0,255],"width":3}},"scaleSymbols":true,"transparency":0,"labelingInfo":null}

When the layer(s) are added can this be used to set the style automatically? If there is no property then leave as the default OL style. I'm not sure how legends work for vector layers - this can be developed later if complicated. I'm not sure what the popup would be for?

[jansule]

When the layer(s) are added can this be used to set the style automatically.

Ok, now I understand. Automated styling based on drawingInfo is not possible without a dedicated style-parser (which we do not have currently). However, it is possible to manually style the layers with GeoStyler UI after importing. This is also what I meant with regard to the popup.

[geographika]

Ok that makes sense. Something to tackle in a future GeoStyler codesprint! It looks like this feature is now complete once the BasiGX pull request is merged. Thanks!