Ways to improve the catalog search function

[robragsdale]

@lukecampbell I am working with @neaptide Sara Haines (UNC-CH and SECOORA) on how the catalog search function could be improved beyond the current built-in features that use the underlying metadata records to find data sets based on geography (i.e. regions) and organization (i.e. RA or Agency). We want to enhance the catalog search feature by giving a user the capability to search on variable names.

Here is a wiki wiki that describes what it is we are pursuing. There are questions we have about how the catalog is currently harvesting metadata. Something we would like to know is what fields (e.g. keywords, variable names) from the services, for example and in particular SOS, are harvested to populate the catalog. Is it fair to say that the fields listed in this example returned from a search in the catalog represents the fields the catalog harvests from the metadata?

[lukecampbell]

I'm glad you bring this up, as a matter of fact we've already started discussing this and it meshes well with another issue I'd like to pursue as early as possible and that is having a proper relational database.

https://github.com/ioos/catalog/issues/178

Right now the database, MongoDB, is not very well suited for related data and associations. Or at the very least, I don't prefer it. I'm looking to migrate to a relational databse like PostGIS and expand the data-model to include this very thing.

Things I'd like to add to specific datasets

• keywords
• standard_name (variable listing)
• units
• better geospatial awareness

[benjwadams]

Seconded. Mongo has support for simple geospatial queries, but to do more heavy lifting you really need a more robust spatial database such as the PostgreSQL + PostGIS stack. Compare http://docs.mongodb.org/manual/reference/operator/query-geospatial/ with http://www.postgis.us/downloads/postgis21_cheatsheet.pdf

[robragsdale]

@lukecampbell is it correct to say that these terms

• keywords
• standard_name (variable listing)
• units
• better geospatial awareness are not being searched on when the catalog is harvesting from the TDS and/SOS services.
  What terms does the catalog search for when it goes out to harvest?

Any update on the status of migrating to a PostGIS database #178?

[lukecampbell]

We won't be upgrading to PostGIS until 4.0 which is late next year at the earliest, I believe.

The catalog harvests a bunch of terms, for DAP endpoints it harvests just about all the metadata it can: all global and variable attributes. For SOS, I think we're using another library to harvest metadata from SOS, I'm not entirely certain off the top of my head all the metadata that gets harvested from SOS.

[robragsdale]

@lukecampbell @neaptide I am reposting this link to Vocabulary Use Cases that were created to show usefulness for refined keyword or domain topic searches. The use cases illustrate search capabilities that I believe we would want developed with the evolution of the catalog. It does not sound feasible that these capabilities could be added to the catalog until later this year or when the catalog backend is upgraded to a PostGIS server. Is that right? I would like to keep these use cases in mind so when the time comes we can revisit how to improve the catalog search function. I will close this post if everyone agrees and we can discuss the catalog search function when the backend upgrade happens.