search

ioos / ckanext-ioos-theme

IOOS Catalog as a CKAN extension
GNU Affero General Public License v3.0
7 stars 14 forks source link

ERDDAP links need to get inserted into ISO metadata from SOS or THREDDS endpoints #115

Closed rsignell-usgs closed 4 years ago

rsignell-usgs commented 7 years ago

We want to see all the available services listed for each dataset in the catalog. But clearly we are not going to get an ERDDAP service endpoint in an ISO record generated by THREDDS or SOS, or get a THREDDS endpoint in an ISO record generated by ERDDAP.

This means the WAF maintainers have some additional work to do: they need some script or method to merge the ISO records created by these services into a single record. Do we have such a tool?

Of course in come cases, custom scripts or methods will be needed to insert things like WMS endpoints from sci-wms or ncwms2 into records.

Can we create a place for such scripts to live, or is there such a place already?

Here's an example from NERACOOS: I search the catalog for "a01" buoy, and return the opendap and SOS endpoints, but not the ERDDAP tabledap endpoint at: http://www.neracoos.org/erddap/tabledap/A01_met_all.html 2016-12-02_5-44-12

mwengren commented 7 years ago

NERACOOS is a good example of this. There is in fact an ERDDAP WAF for NERACOOS registered in the Harvest Registry, but I think there are some content issues with the records generated by ERDDAP, so they are not all making it to the dev-catalog. cc: @ebridger. Here is the relevant link (authorized users only can view): https://registry.ioos.us/records/ZAqasedd44MsWvxeh.

Some tool to merge services from one source or another would be nice, but nothing like that exists that I know of and it may be tricky to develop. How do you associate datasets together, how do you identify where to copy service references from in the ISO metadata since there can be many locations, etc? I don't know if the payoff would be great enough, seems it might require further discussion.

In the NERACOOS case, the tabledap endpoints would be available, if the XML generated by ERDDAP were compliant.

rsignell-usgs commented 7 years ago

@pacioos and I were discussing this last week. Sounds like they have some scripts to do the munging to get a single metadata record. @pacioos, can you provide some examples?

pacioos commented 7 years ago

Actually, if you include ERDDAP links in your TDS catalogs as "viewer" links, then they can get inserted by ncISO as gmd:distributorTransferOptions. The only downside is a generic gmd:protocol of "http" gets assigned. I then have a cron that saves the ncISO output to a file and replaces all the ERDDAP gmd:protocol entries to either "ERDDAP:griddap" or "ERDDAP:tabledap" depending on whether griddap or tabledap appears in the URL. Cheers, John Maurer, PacIOOS

In TDS config:

https://goo.gl/esPzle

Result in ncISO:

http://oos.soest.hawaii.edu/erddap/griddap/SWAN_Oahu_Best.graph http ERDDAP This URL provides a viewer for this dataset. download https://goo.gl/rPwtT0 After script converts gmd:protocol away from default of "http": http://oos.soest.hawaii.edu/erddap/griddap/SWAN_Oahu_Best.graph ERDDAP:griddap ERDDAP This URL provides a viewer for this dataset. download http://pacioos.org/metadata/swan_oahu.html On Tue, Dec 6, 2016 at 5:20 AM, Rich Signell wrote: > @pacioos and I were discussing this last > week. Sounds like they have some scripts to do the munging to get a single > metadata record. @pacioos , can you provide > some examples? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , > or mute the thread > > . >
mwengren commented 4 years ago

Unlikely to implement this. ERDDAP and THREDDS WAFs are managed independently by most RAs and fed into the Catalog/Registry separately, and, unless like PacIOOS they each put in place a mechanism to curate their own metadata records that merge both sources, it'll stay this way.

Closing this since it's rather old and doesn't seem like an issue for the IOOS Catalog to solve.