Get Map view of results working

[ggeisler]

Questions about what is necessary for the Map view of search results to show hits have come from several sources. It's not clear to me what is necessary to get the map view to work for a given exhibit (metadata issue? indexing issue?).

We should clarify the issue when we plan the next work cycle so everyone understands what and how much work is required to get this working, as several exhibits would benefit from having a map view.

[jcoyne]

Is this just placename_coords_ssim? https://github.com/sul-dlss/exhibits/blob/master/app/controllers/catalog_controller.rb#L61

[ggeisler]

Related to #219

[jcoyne]

@mejackreed can you take a look at this ticket and see if you can get us pointed in the right direction?

[atz]

@mejackreed ping

[mejackreed]

Blacklight Heatmap indexing requires a Solr Spatial Recursive Prefix Tree field to be indexed. In default Solr 6.x (no Java Topology Suite installed, e.g. sul Solr cloud) this can be done in two ways:

• x y Syntax. example: "-121.631609 36.688128" For lng lat data only
• CQL ENVELOPE Syntax (minX, maxX, maxY, minY). example: "ENVELOPE(122.934585571, 153.987060547, 45.522888184, 20.422889709)" For bounding box data

See indexing data.

The field then can be referenced in the catalog_controller.rb

Preference is to index bounding box over center point lng lat

[mejackreed]

Sorry all ignoring this project on GitHub to keep down my notifications.. ping me on Slack if you need help with this.

[mejackreed]

For several of the records I looked at (Stanford Archives, Fitch) geospatial metadata was available as bounding boxes.

Examples:

• https://purl.stanford.edu/rh234sw2751.mods checkout the geonames.org uri

<subject>
  <geographic valueURI="http://sws.geonames.org/5350937/"/>
</subject>

Then use the geonames.org API to grab the bounding box: http://api.geonames.org/get?geonameId=5350937&username=demo

<bbox>
  <west>-119.9344</west>
  <north>36.91154</north>
  <east>-119.655</east>
  <south>36.66216</south>
</bbox>

• https://purl.stanford.edu/fw686gh2768.mods using mods cartographic 😢 . I feel like a lot of this data may have problems, I believe @cbeer looked at this a while back.

<mods:subject>
  <mods:cartographics>
    <mods:scale>Scale in English miles</mods:scale>
    <mods:coordinates>(W 18°--E 51°/N 37°--S 35°)</mods:coordinates>
  </mods:cartographics>
</mods:subject>

Need to convert the coordinates from degree minutes seconds to decimal degrees.

[ggeisler]

Currently, the search results map view in Exhibits shows a map but results are never displayed on the map. Also, the item show pages in an exhibit currently do not show any map info. This ticket is for work necessary to improve those issues by incorporating the Blacklight_heatmaps gem to produce several changes when viewing a SUL exhibit (an exhibit that has the appropriate geo metadata, such as Fitch):

• [ ] On a search results page, when you click on the map view icon a map should be displayed with colored boxes representing the geo data of relevant records. For example, for Fitch, searching for Fresno (or doing any search and selecting a value from the Region facet) and selecting the map view should show highlighted locations in California, very roughy similar to this:

• [ ] When you click a colored region on the map, you should get a list of item titles displayed above the map, similar to this:

• [ ] When you click on a title from the list, you should go to an item show page where, in addition to the current content of the show page, you should also see a map with a bounding box (assuming you are using a Fitch item to test), similar to this (note that this example doesn't have the usual show page metadata on it):

---

Note that this work probably depends on #219 so that the required metadata is indexed to Solr. See @mejackreed's comments above for details. The Fitch item @mejackreed referenced in his comment above can be seen in the Fitch exhibit here: https://exhibits.stanford.edu/fitch/catalog?utf8=%E2%9C%93&exhibit_id=fitch&search_field=search&q=rh234sw2751

The Blacklight_heatmaps Readme: https://github.com/sul-dlss/blacklight_heatmaps

Also, the item show page will need work to position and style the map view with the other show page content; ticket #223 will address that.

[atz]

Do we have a legit geonames API username to use? If not, what should I code the indexer to use? I tried it without one, but it is required.

[mejackreed]

"We" as in SUL doesn't have one, but you could create a dummy account and use that. GeoNames isn't really restrictive on this, just restrictive at max requests/day.

If you don't feel like creating one, feel free to use mine, mejackreed

[atz]

Yeah, demo account is already maxed out or disabled on the API. Seems odd we have one URI for a resource geonames, but then have to hit a different server for the metadata about that resource?

To be clear, in the first phase I will be pulling //Subject/Geographic elements from mods and checking for attribute valueURI to match regex: /https?:\/\/sws\.geonames\.org\/(\d+) and then fetching XML using the (digits) ID from api.geonames.org. The target values are in //geoname/bbox.

In the second phase I will attempt conversion of the degree-based cartographic data.

[atz]

Jack referred me to these lines of the heatmap schema. Note this goes against the current naming style where *_bbox maps to solr.SpatialRecursivePrefixTreeFieldType. Instead, it would be more transparent for *_bbox to map to the solr.BBoxField class.

[atz]

@mejackreed I can get the basic syntax of "ENVELOPE(122.934585571, 153.987060547, 45.522888184, 20.422889709)", what are the units of those values? Those look like degrees to me (and our schema has location_rpt in degrees) but your schema that I linked above has distanceUnits="kilometers"!

[mejackreed]

No need to worry about km units here. All of our geometry indexed should be in decimal degrees. distanceUnits are used in Solr to do distance queries, which we aren't doing any.

You should use the distanceUnits that is correct for our version of Solr that we are running. I think default used here should be kilometers

[atz]

Note from conversations today: Some geo objects have more advanced metadata that includes some already processed geonames, like this one:

<extension displayLabel="geo">
  <rdf:RDF xmlns:gml="http://www.opengis.net/gml/3.2/"  xmlns:dc="http://purl.org/dc/elements/1.1/">
    <rdf:Description rdf:about="http://purl.stanford.edu/kv451wx8026">
      <dc:format>image/tiff; format=GeoTIFF</dc:format>
      <dc:type>Dataset#Raster</dc:type>
      <gml:boundedBy>
        <gml:Envelope gml:srsName="EPSG:4326">
          <gml:lowerCorner>-122.7848707 37.8104334</gml:lowerCorner>
          <gml:upperCorner>-122.5829881 37.970422</gml:upperCorner>
        </gml:Envelope>
      </gml:boundedBy>
      <dc:coverage rdf:resource="http://sws.geonames.org/3687919/" dc:language="eng" dc:title="California"/>
      <dc:coverage rdf:resource="http://sws.geonames.org/5370468/" dc:language="eng" dc:title="Marin County (Calif.)"/>
      <dc:coverage rdf:resource="http://sws.geonames.org/8411083/" dc:language="eng" dc:title="Pacific Ocean"/>
    </rdf:Description>
  </rdf:RDF>
</extension>

But because this is not representative of current Exhibits data and has limitations (3 geonames reduced to 1 box), pulling from the <extension> is explicitly out of scope for now.

[caaster]

Here is an example of a published druid that will soon be imported to an exhibit, with geo MD: druid:cw222pt0426

[jcoyne]

Here's the geo xml from cw222pt0426:

<extension displayLabel="geo">
   <rdf:RDF>
     <rdf:Description rdf:about="http://purl.stanford.edu/cw222pt0426"> 
        <dc:format>image/jpeg</dc:format>
        <dc:type>Image</dc:type>
        <gml:boundedBy>
           <gml:Envelope>
              <gml:lowerCorner>-122.191292 37.4063388</gml:lowerCorner>
              <gml:upperCorner>-122.149475 37.4435369</gml:upperCorner>
           </gml:Envelope>
        </gml:boundedBy>
      </rdf:Description>
    </rdf:RDF>
</extension>

[atz]

@jcoyne: My sense is that the level of XML parsing needed for <extension> shouldn't be stuck in spotlight-dor-resources though: it should be pushed down to stanford-mods so that we can be consistent across projects. Thoughts?

[jcoyne]

That's fine by me. I'm mostly celebrating that we don't need to do named entity resolution through geonames. 🎉

[atz]

We don't always have to resolve through geonames, but we still have to have that. Probably more often than not. Even where we have this form, many geonames mapping to one <extension> box is data loss.

[jcoyne]

@atz SUL, may need that at some time, but to resolve this ticket, we only need to make the heatmap view work for cb22pt0426cw222pt0426 (or other druids with exactly this type of geo metadata).

[atz]

@jcoyne: you're confused about the context of the conversation had with Cathy. The example she added were by my request as secondary and additional to the geonames resolution. They are not a replacement for that requirement, and the data in a single <extension> field is never sufficient for multiple geonames.

You have a habit of entering into conversations of my work late in the engineering and then attempting to reduce the scope to something you find easier. I don't mind reducing scope, but it needs to happen before the work is done, like 2 days ago when I explicitly declared the order of implementation, from my reading of this issue.

Note: cb22pt0426 is not a valid druid.

[jcoyne]

I've replaced this ticket with three tickets:

• [ ] #274
• [ ] https://github.com/projectblacklight/spotlight/issues/1521
• [ ] #275 (this may be the same as https://github.com/sul-dlss/spotlight-dor-resources/issues/83)

@caaster indicates that the top two are of much higher priority.