Editor for data source configuration

Aircloak / aircloak

This repository contains the Aircloak Air frontend as well as the code for our Cloak query and anonymization platform

2 stars 0 forks source link

Editor for data source configuration #4031

Open sebastian opened 4 years ago

sebastian commented 4 years ago

Time and time again I see administrators struggling to create their data source definitions.

They are complex to manually setup because they can get really long
The JSON can get quite complex
Defining the keys is also a complex matter

It would be great to offer an interface that assist an administrator in:

configuring the tables
selecting the table type (personal, non-personal)
specifying join keys for the tables

sebastian commented 4 years ago

Could assist in getting place versions of tables for different keys (https://github.com/Aircloak/aircloak/issues/4036).

If there are multiple hierarchies of IDs it could help build and maintain and expose different data source definitions for these (https://github.com/Aircloak/aircloak/issues/4037).

sebastian commented 4 years ago

Arnold tells me they have the problem where they are likely going to need to create a larger number of smaller data sources. The data sources providing slightly differing views to the same data. A set of the tables in these data sources are likely going to be shared. Currently a change to these tables would require the need to manually update all of the individual data sources.

The data source editor could allow the analyst to separate the definition of the data model (i.e. all the tables to could conceivably made available in a data source and how they connect) from a data source. That way you could easily define multiple data sources based on the same data model that all could be updated when the underlying data model changes.

sebastian commented 4 years ago

Arnold tells me they have the need to blacklist certain columns from appearing in certain data sources even though they are in the table. The ideal solution would be for him to have a whitelist of the columns he wants to be part of a table, but this is incredibly hard to maintain using the "queries" feature of a data source model definition. A data source editor could in fact take care of whitelisting for the administrator, making it a manageable task, and thereby also addressing and eliminating the need for adding support for blacklisting of columns as they could just be excluded from the whitelist instead (at least in the context of what Arnold needs in this instance. Blacklisting is still a different feature)

sebastian commented 4 years ago

Jira feature request: https://aircloak.atlassian.net/browse/SUP-36

sebastian commented 4 years ago

This could be useful to render diagrams of how tables relate: https://mermaid-js.github.io/mermaid/#/