Solr facets incorrectly lump WMS services from ArcServers under 'ESRI REST'

[fishytodd]

Link Type detection solution that was provided by Esri is mis-characterizing service links that come from ArcServers. Good with ArcREST, but lumps ArcServer WMS into that facet, so the individual facet numbers are off...

This was originally submitted as 2 topics in one issue, and I'm separating them into separate issues so they can be better tracked. Renaming this one to be more specific.

Ecotrust/wc-data-registry#121

[tchaddad]

Fixed this title so that it reflects the original bug.

[tchaddad]

This is most important if we expose Service type in the UI

[tchaddad]

To accommodate multiple paths for link detection, we need to use "OR" logic in the XSLT files for Geoportal and Solr.

See this KML example for syntax from the Solr config files (/usr/local/tomcat6/webapps/gc/WEB-INF/classes/gc-config/xmltypes) :

  <xsl:for-each select="//gmd:identificationInfo/srv:SV_ServiceIdentification/srv:serviceType/gco:LocalName[contains(translate(.,'KML', 'kml'),'kml')] | //gmd:identificationInfo/srv:SV_ServiceIdentification/srv:serviceType/gco:LocalName[contains(translate(.,'KMZ', 'kmz'),'kmz')]">
        <field name="dataAccessType_ss">KML</field>
        <field name="url.kml_ss">
            <xsl:value-of select="//gmd:identificationInfo/srv:SV_ServiceIdentification/srv:containsOperations/srv:SV_OperationMetadata/srv:connectPoint/gmd:CI_OnlineResource/gmd:linkage/gmd:URL[contains(translate(.,'KML', 'kml'),'kml')] | //gmd:identificationInfo/srv:SV_ServiceIdentification/srv:containsOperations/srv:SV_OperationMetadata/srv:connectPoint/gmd:CI_OnlineResource/gmd:linkage/gmd:URL[contains(translate(.,'KMZ', 'kmz'),'kmz')]"/>
        </field>
    </xsl:for-each>

Approach will be similar for the Geoportal XSLT files

[tchaddad]

This will resolve uncounted WMS from OCA.

WMS links from ArcServers are currently being detected correctly, but are also being double counted under ESRI REST (somewhat by design).

At the moment, ESRI rest is detected using the term "Mapserver", which is limited to detecting "Map Services". If we want to detect ESRI feature services we will have to add more logic to detect "Featureserver" or "Imageserver" as appropriate.

[tchaddad]

Hopefully this comment will help clarify the situation. Not all of this may be fixable, and/or this issue may have to be further split:

• The solution provided originally for link type detection of what we are labeling ESRI REST services in the UI is reliant on looking for the keyword "Mapserver" in the URL

This is problematic for at least 2 reasons:

• The keyword "Mapserver" only exists in URLs for what ESRI calls "Map Services". It does not exist in URLs for ESRI "Feature Services" or ESRI "Image Services".
• When a host is serving WMS services from an ArcServer, the ArcServer uses the same "Mapserver" in the root URL
• as a result, when Solr counts service types, it double counts WMS services from ArcServers as ESRI REST, and it misses (i.e. does not count) Feature services or Image services.

This second point was/is the cause of the original issue posted above. In looking at the first set of "Service Type" numbers in the Solr Link type facets (http://wcga-vm01.sdsc.edu/gc/), something seemed off. About half the WMS records are missing from the count, and records from ArcServers were appearing in both "WMS" service type and "ESRI Mapserver" service types. In digging into it with Christine White, we discovered a few things:

• with respect to ESRI REST, Solr is performing exactly as expected given what we are asking it to do. It's just that the results are misleading. WMS from ArcServers will always be double-counted as ESRI REST as long as we use the keyword "Mapserver".
• In addition, a compounding problem we discovered is that Solr was under-counting service types of potentially several kinds due to not accounting for various metadata paths where links are stored. This happened to include all the records for the OCA which made it very noticeable. This is a new issue, and can be resolved by accounting for multiple paths in Solr link detection configuration files, using the KML example above.

The problem with WMS and "Mapserver" double-counting made us aware that the same issue likely exists for WFS and ESRI "Feature Services" and WCS and ESRI "Image Services" but it has yet to come up from users because we have few examples of these types yet, and we are not showing counts for the ESRI versions at all. However these types are expected to become more common, especially if we have more members who use ArcGIS Online - which primarily allows users to create feature services.

[tchaddad]

In terms of action on this issue, I suggest that for now, we:

• not fix the ESRI Mapserver issue (it is misleading, but not really a bug)
• fix the paths problem in the Solr link detection so that Solr is picking up services in all the places that they are being stored.

Will try to provide the additional link path locations that need to be incorporated in the Solr config. They are documented in the Geoportal config files, just need to be moved over to Solr as well...

[tchaddad]

I believe we've accomplished the suggested actions from the April 21 comment above. Adding a "closable" tag to track.

[tchaddad]

@fishytodd just wanted to check with you to see if you agree that we can close this one? I assume you had something that was leaving it open?