Open degoldner opened 1 year ago
@anmolsgandhi more details what is being planned here would be good
This issue has been added to v2.0.0 for further investigation. At the moment there are no details on the actual implementation because we are still gathering requirements. However, the main goal is to allow a more configurable definition of the lifecycle of a dataset. @degoldner and his team have implemented an extra stage for datasets plus more customization on permissions. Once they are imported/created they need to be published before being shareable in the catalog.
In this issue we want to dive deep into their customizations and investigate/implement a way in which we could add that feature to data.all v2 in a generic way that could serve all customers. Once we start the discussions I will make sure to keep you in the loop @zsaltys.
@degoldner we are studying this feature as a candidate for our next releases. Can you provide more details, screenshots, diagrams on your design?
This issue will be closed soon as stale issue
Is your feature request related to a problem? Please describe.
In order for the data.all product to be adaptable by different users with different requirements we need to make the features be modularizable. This was greatly already achieved on the level of datasets, data pipelines and further features.
Another important layer which would require modularization possibilty is the governance layer where we define our permission model and the business workflow for governance, like data curation and data share request approval. The requirements for the governance layer might differ from user to user.
Additional context For our example we adapted the data.all code (not yet on new modularized version) by introducing a RBAC model with specified roles and permission. Also we added a curation process which is in between dataset import and dataset publishing to the data catalog.