Improved notifications

[niemyjski]

The goal of notifications is to keep you informed without overwhelming you with too much information. We could also be better by sending out emails from different accounts (notifications@exceptionless.io vs billing@exceptionless.io, etc). This could help you to filter emails out and find what you are looking for.

It would also be nice to send notifications in these use cases in addition to the use cases we already support:

Email notifications

• [ ] Ability to opt out of new 404 messages
• [ ] A event marked as fixed, user comments on an error, adds a reference link.
• [ ] A projects event rate is increasing abnormally.
• [ ] Ability to control overage emails.
• [x] Keep track of project stats so we can show nice charts as well as overage info in daily summaries
• [x] Ability to quickly opt out of email notifications
• [ ] Ability to configure daily as well as the ability to not send it if nothings occurred during the last 24 hours [notification times] (https://github.com/exceptionless/Exceptionless/issues/245)
• [x] Additional information on all emails (tags, descriptions, user identity).
• [ ] Ability to configure rules based on types, tags, time of day, number of exceptions, or environment.
• [x] Ability to snooze notifications for a period of time.
• [ ] Send a periodic digest email based on activity
• [ ] Don't send as many emails if there hasn't been any changes in activity. Example a user hasn't configured there project shouldn't get a daily notification saying they are Exceptionless every day. Maybe the best option is to allow the user to choose between a daily or weekly summary.
• [x] Emails on billing failures/status
• [ ] Improve billing email receipts and view payment invoices
• [x] Email when being throttled or plan limit reached
• [ ] Add ability to set what time daily notifications go out.
• [ ] Add ability to see notifications for all event types like log errors.
• [ ] Notified when bots are detected / security scan on project level....

In app notifications

• [ ] Notify the user when events have been hidden due to reduce noise setting
• [ ] Notify the user when the queue is behind or other system related issues via in app message.
• [ ] Would be great to notify in the ui as well if emails are unable to be sent out.

Third Party Integrations

Provide a easy way to manage third party integrations. Currently these integrations can be done with zapier. We should define what the default message should look like.

• [x] Slack
• [ ] Hipchat

  Feedback

Please leave your feedback in the comments below. Here is a bulleted list of feedback so far:

• @webcoda suggested that it would be really nice to be able to send an alert to any user (even if they are not part of the system).

[HenrikHoyer]

Please add an "daily/weekly/monthly" option

[HenrikHoyer]

Please remove the "congrats, no event was logged" mails - they are just noise

[niemyjski]

@ejsmith what do you think about not sending dailies if there are no events? Could lead me to think that notifications are not working or we have not yet configured our project. We do have project is configured flag. Maybe we add configuration required emails and option to not send.

[HenrikHoyer]

I want to receive exceptions from you, not the normal case :-) All monitoring systems we use only send notifications on exceptions/deviations - none are sending "everything ok mails".

But if your current users are depending on these "everything ok" mails, then add a "send summary mails even if no exceptions has been logged" flag (imho with default = no)

[davidkeaveny]

HipChat integration would be useful for certain critical events, such as a new type of error. As long as you can just the dial on noise vs signal then you're OK.

[ejsmith]

Yeah we absolutely have to do a good job with our signal to noise ratio

[jakejgordon]

The biggest thing I've noticed so far in the email notifications is that if there is a new exception I'd rather have the full stack trace in the email so I can quickly decide whether I need to go in and investigate or whether I should ignore it (for now). Of course the "new error" part is the hard part but I don't have any reason to think you'd mess that up :smile:

[niemyjski]

1. Add snooze actions to stack
2. make it easy to trigger snooze actions from emails
3. Add stack trace to email.
4. Keep track of emails and stop spamming emails. It's crazy. We gotta figure out how much were sending and stop sending streams of emails. Emails should be important!

[niemyjski]

We need to look at our own email inbox and see how we can ditch 90% of our emails.

[niemyjski]

For error emails we should have the link open to the stack trace tab for quick viewing as well as add the stack trace to the emails as previously noted.

[niemyjski]

Here is what our current thought process is for working notifications starting today (finally).

• [x] We update our email templates and make them easier to edit and add content.
  • [x] We are thinking about using https://mjml.io or (https://github.com/leemunroe/responsive-html-email-template combined or https://github.com/zurb/foundation-emails with https://putsmail.com/inliner) to quickly generate valid markup from simple html.
  • [x] Then we can test our emails using a service like [litmus] (https://litmus.com/checklist/emails/public/d432046) and mailinator
  • [x] we need to figure out a way to easily update our razor templates or find a better way to generate our emails from the email templates above.
• [x] Next, we are going to add additional information as stated above to emails.
  • [x] Add tags
  • [ ] Stack traces and more
• [ ] We need to identify ways to reduce noise of emails
  • [ ] Identify sources of noise by looking at all our current inboxes.
  • [ ] Look into doing anomaly detection / perculators?
  • [ ] Look into what it's going to take to do things like rules etc..

There is a lot to do but here are the first steps. We want to do everything on this issue but we feel like these are the first steps that need to occur..

[ngarnier]

Hey @niemyjski, seems like you decided to go with FFE. I'd really love to know more about what made you go for this framework rather than MJML 🙂 .

Cheers!

[niemyjski]

@ngarnier I think both FFE and MJML were in a close running and have a large community base and both are great solutions. We went with FFE because we felt it was a bit cleaner syntax (with inky) overall and more aligned with what we are using (lots of people using handlebars already with zurb, which we are going to use server side to do a second pass on the template). It felt like it had less of a learning curve. For example doing partial templates is pretty straight forward in FFE and I'm not sure I even seen an example of it in your docs . (Edit needed to look further: https://medium.com/mjml-making-responsive-email-easy/tutorial-creating-your-own-mjml-component-d3a236ab7093)

I do think MJML is a bit sexier, and we have some friends who use it in their react projects (it’s how I heard about it). I'm wondering If our project was in react, if we would have leaned more to pick MJML.

[ngarnier]

Thanks a lot for the feedback @niemyjski!

Regarding partials, it's done with mj-include. If you're talking about templating languages, it's true we prefer not to implement our own but rather enable users to use the one they want.

About React, we're actually dropping it in the next version (see the MJML 4 announcement) to improve performances, so it shouldn't be the main argument. Rather than using React, the JSON format of MJML makes it really easy to build/manipulate a responsive email programatically.

Thanks again for the feedback and feel free to reach out if you have any question regarding MJML!

[niemyjski]

Thanks for the update and I'll follow the project and v4. For us, having our server side stuff in .net, we'd have to prerender the templates or rewrite our mail job to be a node app (doable but a pain as we'd have to hook into our queues). So we wouldn't get the perf benefits of vnext, but it does sound like it's going to make your lives easier and less of a hack transforming the html :).

One of the things I think FFE does better at least out of the box is having convention / structure for the body of the email and then a partial for the content. I think if you went this route in your samples it would go a long way (I remember looking at your sample repo and you had thrive header, thrive body and thrive something and it was confusing as they all seemed to contain the same content...) Most of the content in the samples is slightly duplicated from one to the next. You're probably always going to have the same header and footer throughout your app so why specify it multiple times. It was nice to dig into and just update the main body in FFE. Please correct me if I'm wrong, I spent an hour or two looking at both :)

Also, I greatly appreciate you reaching out to us and having this conversation. It says a lot about your team!

[ngarnier]

Oh yes you're right about having to run your own node app. Another option would be to use the MJML API but it does add a dependency to your email system.

Regarding the thrive examples, I see. They were meant to illustrate the different advancement steps of the email for a tutorial rather than how to use partials, but it makes sense that it might have been confusing now. We'll work to improve the documentation to reflect this better as it's really easy to do with mj-include.

I'm glad you appreciate it, I didn't mean to overflow your issue's thread but rather to learn what we can do better, so thanks again for your answers!

[niemyjski]

We need to add rules for overage emails so you can opt out of them.

[niemyjski]

We should also look into something like https://image-charts.com for awesome charts in emails

[niemyjski]

@ejsmith we should add the count for each item in the most frequent list on the summary email. I like that I can see the list of the top issues for the day, but seems like I would want to know how many times they happened. I think we should do top 5 and then under that put “X other issues occurred N times” and that is a link to view the list of all the stacks that happened that day.

[niemyjski]

We should send an email the first time you are throttled every day.

[niemyjski]

We need the ability to opt out of new 404 messages as well.

[niemyjski]

"I'd like to request a feature enhancement. I would like to request the ability to only receive the daily project summary emails if there are items logged. I dont' want to get emails basically saying "Nothing to report today.""

[niemyjski]

"Somewhat. That is mostly how I currently have it set up. We are not marking anything critical or fixed at the moment so I only have new issues set up. I agree with the noise, but it is easy to miss something if it does not get addressed the first time. I wish there was some sort of expiration where if you still have the same issue after X amount of days it is treated as a new issue again. I just don’t want something to get missed and at the same point I don’t want developers to be required to keep exceptionless up on a tab at all times. I was just trying to think of the best way to not miss anything. I have a separate chatroom setup just for exceptionless to limit the noise as well."

[niemyjski]

+1 hipchat

[niemyjski]

Added: ability to set what time daily notifications go out.