Slack Notification with error description

pattyrocks commented 3 years ago

Describe the feature

At the company I work for there's been some issues with identifying what is causing job warns/ errors fast and notify the people responsible for. Today, we use the Slack integration to message an alerts' group where everyone in the team follows, however, every time there's a warn/error the whole team has to stop, open the notification and read the logs. Most of the time only one person out ofthe whole team should have done it and instead, the whole team had to stop to do the check.

Describe alternatives you've considered

I am not a specilist in that matter, so I am here looking for help to find a viable alternative so we can quick see the log information, ideally in the slack notification, but any other mean of achieving it would be appreciated.

Additional context

-

Who will this benefit?

Everybody who uses Slack notifications or job notifications as a trigger to take action

Are you interested in contributing this feature?

It's not my area but I'm definitely willing to help on it.

jtcohen6 commented 3 years ago

Thanks for the thoughtful write-up @pattyrocks! Sorry for the delay in getting back to you.

I'm curious to hear a bit more about what you're thinking. It sounds like you're looking for more details in the Slack message, such as information like:

How many models/tests/etc failed
The names of the failing models/tests/etc

In the very simplest version, the Slack notification could include the "summary" lines from the dbt Core CLI output:

Completed with 1 error and 4 warnings:

Failure in test my_failing_test (models/some_models.yml)
  Got 7 results, configured to fail if != 0

  compiled SQL at target/compiled/.../my_failing_test.sql

Warning in test my_warning_test (models/some_models.yml)
  Got 396 results, configured to warn if != 0

  compiled SQL at target/compiled/.../my_warning_test.sql

...

Done. PASS=495 WARN=1 ERROR=4 SKIP=0 TOTAL=500

Ultimately, though, I think we'll want something more structured, more powerful—and that the majority of the lift here will be in dbt Cloud. To start, it should involve surfacing information already included in dbt Core's artifacts (especially run_results.json).

That being said, I'm wondering if there's fruitful overlap with other proposals, which would tie into other dbt Core features or potential changes changes:

https://github.com/dbt-labs/dbt/issues/3436: specifically, notify the owner of a test when it warns/fails (rather than notifying everyone)
Exposures have owners. If any source, model, or test upstream of an exposure warns/fails, the owner of that exposure should be notified

Does that reflect how you're thinking about this? Anything on your mind that I might be missing?

pattyrocks commented 3 years ago

Thank you so much for this detailed insight. I believe your suggestion including "summary" into the Slack message would be enough for now. However, the robust suggestion would definitely help even more.

It sure is related to issue you mentioned, however even in the scenario that it is in place, a slack notification more detailed could also improve the experience.
I wasn't aware of 'Exposures', I just checked the documentation and it can definitely help me

With all this, to solve my problem, I believe adding the summary to the slack notification would be perfectly combined with the exposure email notification. Do you have any documentation on how to add the "summary" to the Slack notification? Is this feature already working?

jtcohen6 commented 3 years ago

@pattyrocks I'm glad to better understand what you're after. Sorry for not being clearer in my previous message—this isn't a feature yet, though I could imagine us adding support for it in the future.

Since this will need to be a feature in dbt Cloud, rather than a code change we can make in this repo, I'm going to close this issue. When the Cloud teams can prioritize work on notifications, this will be really solid feedback to consider—thanks again for writing it up!

dbt-labs / dbt-core