WCGA / West-Coast-Ocean-Data-Portal

bugs and fixes for the geoportal back end and UI front end of the WCODP
1 stars 0 forks source link

Strategy for Harvesting DCAT #70

Closed cybersea closed 4 years ago

cybersea commented 6 years ago

The new ESRI Open Data sites, like the one at the State of Washington, http://geo.wa.gov, can be accessed using DCAT, like so: http://geo.wa.gov/data.json See ESRI Docs on Federating to CKAN

It looks like Geoportal Server v.1.2.4 can support DCAT output from the catalog, but not necessarily harvest from other catalogs. Likely need to upgrade to more recent version of 1.2.x https://github.com/Esri/geoportal-server/wiki/Customize-DCAT-output https://github.com/Esri/geoportal-server/tree/master/support/dcat

Looks like DCAT harvesting was added in 1.2.5 With Enhanced support in 1.2.6 And even more enhancements in 1.2.7:

Or, transition to v.2.x which has a new architecture. The harvester and the catalog are no separated into two products. There is no database backend and it uses Elasticsearch 5.x for Faceting http://proceedings.esri.com/library/userconf/proc17/tech-workshops/tw_438-26.pdf https://github.com/Esri/geoportal-server-catalog https://github.com/Esri/geoportal-server-harvester

cybersea commented 6 years ago

Sent an e-mail to Marten Hogeweg asking for confirmation that v.1.2.x could be used to harvest DCAT. Received this reply.

hi Allison,

you should be able to take their dcat file: http://geo.wa.gov/data.json and use the new harvester (https://github.com/Esri/geoportal-server-harvester) to achieve this.

PS: in 2016 we released geoportal server 2.x (https://github.com/Esri/geoportal-server-catalog), a complete rewrite using elastic as backend, no rdbms, and no more JSF. in return you’d have faceted search built-in (and configurable). here is a sample site: http://geoss.esri.com/geoportal2/#

kind regards,

Marten

cybersea commented 6 years ago

Key Question: Can DCAT records be selectively harvested?

cybersea commented 5 years ago

Will be upgrading to v.1.2.9

tchaddad commented 4 years ago

DCAT harvesting works now, but selective harvesting remains an open topic.