data-dot-all / dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
https://data-dot-all.github.io/dataall/
Apache License 2.0
232 stars 82 forks source link

Metadata Form Enforcement #1668

Open SofiaSazonova opened 4 days ago

SofiaSazonova commented 4 days ago

Enforcement

Metadata forms can be obligatory to fill in on different levels. User can select the metadata form and entity types, that should have this form attached. Enforcement affects selected entity types on all lower levels hierarchically.

Table E.1. Metadata form enforcement rule

Field Description Type Possible Values
Metadata Form Identifier of metadata form, that is enforced String URI of existing form
Metadata Form Version number of version, that is enforced Int
Level Level of affected entities. All lower levels will be affected. String From table E.2
Home Entity Uri If level is not global, the Uri of the entity under which the MF is enforced String
Entity Types Entity type affected by this enforcement Array [String] Identifiers of data.all entities types (table E.2)
Severity String Obligatory/Recommended

Table E.2. Metadata form levels’ hierarchy

meta levels

Who can enforce:

So in summary, enforcement capabilities cascade along with administrative privileges in the hierarchy. Global admins have full control, org/env admins can enforce for their sphere and below, dataset admins for the datasets and items in it, and share requesters and approvers for a specific share.

How we enforce?

  1. When MF is enforced, but entity misses it, on entity page there is a HUGE reminder about it.
  2. If MF is mandatory, than NO actions are allowed until required MF is attached | 2.1. Orgs:

2.2. Envs

2.3. Datasets

SofiaSazonova commented 4 days ago

MF DB

SofiaSazonova commented 3 days ago

I think, we don't really need Item level. So, I'll remove it

rbernotas commented 2 days ago

@SofiaSazonova , how will this look if as a data.all admin, I want to enforce a Metadata Form for all datasets in data.all? Using Table E.1 as an example:

Metdata Form: the identifier of the metadata form to enforce Version: the version of the form I'm going to enforce Level: Global Level? Home Entity: N/A Entity Types: Dataset Severity: Obligatory

Is this correct? In this case, the Form will be required for all Datasets in data.all? Is it then also required for all Buckets/Folders/Tables/Shares for all Datasets? Or would that only happen if "Entity Types" was "Dataset, Bucket, Folders, Tables, Shares"?