data-dot-all / dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
https://data-dot-all.github.io/dataall/
Apache License 2.0
226 stars 82 forks source link

Send alerts when share verifier ECS task fails #1299

Open TejasRGitHub opened 3 months ago

TejasRGitHub commented 3 months ago

Is your idea related to a problem? Please describe. Currently, the ECS task for share verifier runs on a schedule. When the share verifier service fails it doesn't send any alerts which will inform user about the failure.

Describe the solution you'd like Send an alert in the form of an email, etc to inform the user about failure while running an ECS task.

P.S. Don't attach files. Please, prefer add code snippets directly in the message body.

noah-paige commented 1 month ago

Hi @TejasRGitHub - curious to hear more about what you had in mind here...

Are we talking about notifications for when the schedule share verifier task determines a new share item as unhealthy? Or are we talking about when the actual ECS Task fails when running with some type of error

If the former (notifications when state change to unhealthy):

If the latter (notifications on failing ECS task):

I would think you more so are discussing when state changes to Unhealthy but please correct me if I am wrong!

Curious to hear your thoughts here and how you would want data.all notifications to be handled for this use case

TejasRGitHub commented 1 month ago

Hi @noah-paige , Thanks for the taking a look at this issue.

I was actually talking about the latter. I know that for share-manager an alert is raised if something fails. I think this function is missing the share verifier ECS task.

Can you tell me where these alerts from share manager are sent ? Are they sent to an SNS topic ?