As far as provenance is concerned, DataObject and Dataset are both instances of something called an Artifact, with an associated type. This reduces code complexity significantly.
DataObjects are now stored in Mimir.
DatasetHandle.row_count was a bit of a hack in Mimir. DatasetHandles no longer have a notion of the row count, forcing code that requires the row count to explicitly issue the (heavyweight) command to get it.
A few other improvements to the Vizier/Mimir interface.
I removed workflow scope caching (Module.datasets and Module.dataobjects). Apart from the fact that it's super easy and fast to reconstruct it from the provenance, this is mutable state that doesn't belong in the otherwise immutable Module.
This PR started as a fix for https://github.com/VizierDB/web-ui/issues/236 but ended up rolling in a mountain of housekeeping tasks. I still want to add a few unit tests, so this is still a draft, but the bulk of the changes are in. Note that this PR requires a corresponding update to Mimir-API (https://github.com/UBOdin/mimir-api/tree/add-dataset-properties), which itself requires a few minor patches to Mimir-Caveats (https://github.com/UBOdin/mimir-caveats ; Use sbt publishLocal to get 0.2-SNAPSHOT)
Highlights include:
DataObject
andDataset
are both instances of something called anArtifact
, with an associated type. This reduces code complexity significantly.DatasetHandle.row_count
was a bit of a hack in Mimir. DatasetHandles no longer have a notion of the row count, forcing code that requires the row count to explicitly issue the (heavyweight) command to get it.Module.datasets
andModule.dataobjects
). Apart from the fact that it's super easy and fast to reconstruct it from the provenance, this is mutable state that doesn't belong in the otherwise immutableModule
.