search

data-dot-all / dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
https://data-dot-all.github.io/dataall/
Apache License 2.0
228 stars 82 forks source link

Automated Share Correction with Re-Apply Share ECS Task #1285

Closed anushka-singh closed 3 weeks ago

anushka-singh commented 3 months ago

Is your idea related to a problem? Please describe. We have Share Verifier ECS task scheduled to run weekly. Occasionally, this leads to shares being marked as unhealthy. Thankfully, the 'Re-Apply Share' button proves to be a valuable resource, resolving most of these issues upon activation.

Describe the solution you'd like There is a need to introduce a new ECS task specifically for re-applying shares. This task can execute after the Share Verifier task, allowing for automatic correction of shares without the need for manual intervention.

P.S. Don't attach files. Please, prefer add code snippets directly in the message body.

dlpzx commented 1 month ago

This is a great idea that does not require a lot of additional efforts on top of what we have already implemented :) Are you planning to take it?

noah-paige commented 4 weeks ago

Hello - we have opened a PR that adds a step to the share verifier task that will also start the reapplier task to try and re-apply/fix for any unhealthy share items after verification of all share items is complete

Currently the implementation is default behavior when the verifier task is triggered - please let me know thoughts after taking a look at the PR

@anushka-singh @zsaltys

anushka-singh commented 4 weeks ago

Added a comment on the PR