search

ioos / service-monitor

A web based catalog of IOOS services and datasets
http://catalog.ioos.us
6 stars 13 forks source link

Changes relationship of services to harvests #460

Closed lukecampbell closed 7 years ago

lukecampbell commented 7 years ago

Before this PR, a service had a strong one-to-one relationship with the URL of the service. We said, logically, a service was a service endpoint.

With this change, services are essentially metadata labels associated with a URL and services have an optionally many to one relationship with the service URL.

The problem then becomes what to do about harvests? You don't want to harvest the same endpoint a hundred times if it has a hundred services defined (GCOOS). You want to harvest only once, but then how do the other services show up?

The solution in this PR is:

When harvests are queued, only harvests for distinct URLs are queued. Once the harvest is complete, each service with that URL gets a harvest object that says the same thing as the parent harvest.

Each service URL is harvested only once, all services are updated.

lukecampbell commented 7 years ago

This PR will fix https://github.com/ioos/catalog/issues/22