Add sensitivity and redacted fields

moj-analytical-services / etl_manager

A python package to create a database on the platform using our moj data warehousing framework

21 stars 8 forks source link

Add sensitivity and redacted fields #120

Closed calumabarnett closed 4 years ago

calumabarnett commented 4 years ago

This PR will:

add a sensitivity property to column and table metadata
add a redacted property to column metadata
update tests
resolve formatting issues

The unit tests will not complete successfully as there are changes to table_schema.json that are not yet present in the online version. Tests should pass if https://github.com/moj-analytical-services/metadata_schema/pull/5 is merged.

calumabarnett commented 4 years ago

I've changed it so that the table sensitivity can never be set manually but can still be queried as a property. The property will be set using a private update method based on the column sensitivities.

MrAlecJohnson commented 4 years ago

This looks good to me.