stigbd / data-catalog

Experimental repo to investigate Backstage as a data-catalog
0 stars 0 forks source link

Create meta-data structure to hold meta-data about datasets #2

Open stigbd opened 1 year ago

stigbd commented 1 year ago

Same way as openapi-files is related to catalog-info.yaml

First attempt may be inline.

Attributes must include:

Format should be json.

eidsvold commented 1 year ago

Do we need a separate openapi-like file to hold datasets' metadata? Could it be part of the catalog-info entity? E.g.

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: example-dataset
  description: An example of a dataset
spec:
  type: dataset
  lifecycle: experimental
  owner: datascience
  system: parcelhub
  distribution: parquet
  providesApis: [example-rest-api]

It seems to me that we have to customize the CatalogEntityPage to display custom fields, if we don't create our own dataset plugin.