search

data-dot-all / dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
https://data-dot-all.github.io/dataall/
Apache License 2.0
236 stars 82 forks source link

Review cleanup/implement Subscriptions in Environments #166

Open dlpzx opened 2 years ago

dlpzx commented 2 years ago

Is your feature request related to a problem? Please describe. Currently data.all creates Subscriptions at the environment level where data producers and consumers can subscribe or publish to an SNS topic.

Describe the solution you'd like I would like this feature to be extended. I would like subscriptions at team level or dataset level. As a data consumer I want an easy way to subscribe to a dataset = get notifications in case of changes in the data.all dataset object, tables...

Describe alternatives you've considered At first sight it looks like we could move the subcription object from environment to dataset. Dataset owners and successful share requesters could subscribe to the topic of the dataset if it is enabled by the dataset owners.

We need to review this idea in case of SNS topic limitations and quotas. Maybe we end up creating too many topics.

dlpzx commented 10 months ago

In #291 the notifications received by requesters and approvers during the sharing workflow are improved 🥇 So this issue is partially solved by the PR. However there is still 1 pending task: decide what to do with the SNS code from environment stack/Subscriptions ---> at the moment there is an SNS topic that gets created but it is not an end-to-end feature and customers are not using it. I will rename the issue accordingly