support for caom2 metadata provenance in model

opencadc / caom2

Common Archive Observation Model

GNU Affero General Public License v3.0

5 stars 11 forks source link

support for caom2 metadata provenance in model #66

Closed pdowler closed 4 years ago

pdowler commented 6 years ago

It would be nice to track which software and version was used to generate metadata for an observation. This is especially true when sharing metadata between sites.

pdowler commented 6 years ago

Thoughts, in no particular order:

need to decide if metadata provenance is only of interest to operators and thus something like the entity attributes (id, lastModified, metaChecksum, etc) or if it is possibly of general use.
currently, most of these are visible via TAP, ID values are only useful as join keys in queries, lastModified is useful to anyone looking for recent changes, maxLastModified are not visible, and metaChecksum(s) are visible but probably not useful.
maxLastModified values should be visible via TAP so that a caom2 observation can be completely extracted from a TAP service (eg the caom2-tap library can do it except for this)

pdowler commented 6 years ago

What is the right cardinality: one per observation? plane? artifact?

Plane.creatorID is a product identifier for the metadata creator and is preserved when metadata is shared/harvested to another site... it is a sort of metadata provenance. Maybe something that sits beside that field?
Given that operations could update/generate plane metadata and add it to an existing observation without modifying the observation or other plane(s), this might have the correct cardinality....
Probably Artifact would probably be the smallest entity that would be modified separately...

pdowler commented 5 years ago

sounds like extending CaomEntity with a string field would solve this generically