Add alerts to integration packages

alexfrancoeur commented 3 years ago

This is more of a placeholder than anything. There are lots of details to to work out still, but in 7.x to support 8.0 migration requirements we'll most likely be adding the ability to import / export alerts and actions (https://github.com/elastic/kibana/issues/50266). There is no pressure to prioritize, but at some point we'll probably want to think through the types of alerts we could ship with each integration and the default configurations available. This would not only provide an out of the box experience for analysis, but for making the data actionable as well.

In self managed environments, users will need to set up a connector but we may be able to ship with "in Kibana notifications" by default. In Cloud, we're shipping preconfigured connectors with their email address I believe.

Again, not a priority, but once we have the ability to import / export alerts - we should start thinking through the ideal integration UX. It's a powerful experience shipping rules / alerts out of the gate when onboarding. Pingdom does a good job of this for synthetic uptime monitoring. You onboard, add your URL, immediately start monitoring availability for your endpoint and receive alerts if availability drops. Soon, we should have all the building blocks in place for a similar experience.

cc: @arisonl @mikecote @hmnichols @tbragin @mostlyjason @ruflin

mostlyjason commented 3 years ago

Thanks Alex I added this to our roadmap, but no specific release yet

elasticmachine commented 3 years ago

Pinging @elastic/integrations (Team:Integrations)

ruflin commented 3 years ago

Can we move this to the package-spec repo? This is where discussions around adding support for new asset types should start: https://github.com/elastic/package-spec/issues/108 https://github.com/elastic/package-spec/issues/27

tbragin commented 3 years ago

@alexfrancoeur we are considering investing in this down the road, but it is likely going to come as an implementation detail of a user-focused effort to introduce alerting libraries at scale (e.g. https://github.com/elastic/observability-dev/issues/782 )

What's the reason you're bringing it up? Is there a dependency on this functionality within the Kibana Platform Team or Alerting V2 effort we should keep in mind?

cc @mukeshelastic @sorantis @cyrille-leclerc

alexfrancoeur commented 3 years ago

Apologies for the wrong repo, thanks for moving. Also, thanks for sharing the alert libraries initiative @tbragin, that approach makes sense.

I bring this up in a more opportunistic fashion than anything and wanted to create a placeholder for any backlogs if the team agreed. As part of the migration requirements for 8.0, Kibana alerts and actions will likely be importable and exportable through Kibana's saved object management. Which they are not today. With this capability on the horizon, we can now support use cases such as shipping alerts as json blobs that are part of a package, similar to dashboards, visualizations and other Kibana assets. I know we discussed potentially going down this route with Security rules recently. If this is not the approach the Observability or Security teams would like to take, we can close this issue out.

tbragin commented 3 years ago

Thanks for the context, Alex. Good to know about upcoming ability to import / export alerts as saved objects! I think it makes sense to keep the issue open - we would like to introduce more prepackaged alerts in the future.

cc @cyrille-leclerc @alvarolobato

masci commented 3 years ago

ruflin commented 2 years ago

@vinaychandrasekhar I think it is time to revive this effort to bring alerts / rules to packages together with SLO: https://github.com/elastic/package-spec/issues/435 Each package developer must be able to provide predefined alerts / rules for the data that is shipped that users can easily modify for their own needs. Simple example: Send alert when disk usage is > 90% should be part of the system package.

kpollich commented 3 months ago

@nimarezainia - Assigning to you for prioritization. This has been asked for consistently over the last few years, but I'm not sure when the right time to revive it will be.

flash1293 commented 3 months ago

The need for this also came up in the context of the observability onboarding initiative. I think for the current plans (by October) we can get by the requirement, but it would be a stopgap until this functionality lands.

Happy to sync, I have some thoughts how this could look like

nimarezainia commented 3 months ago

The need for this also came up in the context of the observability onboarding initiative. I think for the current plans (by October) we can get by the requirement, but it would be a stopgap until this functionality lands.

Happy to sync, I have some thoughts how this could look like

@flash1293 would you mind providing the use case in context of o11y onboarding? what is the workflow you envisage? Any customer feedback as to why this is required? (TIA)

flash1293 commented 3 months ago

@nimarezainia

The workflow would be the following:

User onboards data using an integration (as part of this, the integration is installed) - this either happens through the integrations page or one of our guided onboarding flows
As part of the integration, alerting rule "templates" are shipped as assets. On install of the integration, these are not materialized as actual alerting rules, they are simply part of the integration package
On the integration detail page, on a new central "recommendation page" we are currently working on, and where it makes sense in context in the observability solution, we prompt the user to use these alerting rule templates if they didn't use them yet.
When the user chooses to do so, we pre-populate the "create alerting rule" flyout form with the alerting rule template from the integration. The user is free to tweak settings, we simply provide them a starting point
On save we store the information what template the user used to create the rule as well

This is a bit different from other assets we ship already - instead of making them available as a "managed" read-only saved object, we only provide a template here for the user so they don't have to fill in all the fields of the alerting rule from scratch, but we provide them a recommended set of configs that can be adjusted to their needs.

cc @akhileshpok

nimarezainia commented 3 months ago

thanks Joe. Just to confirm: Obviously the alerting rule template is generic in this case and the user supplies the fields to alert on, perhaps also the connectors they want to deploy.

(I would like to see if this "create alerting rule" template, can be extended also to apply to agents and their statuses. Invoked from Fleet UI)

flash1293 commented 3 months ago

Obviously the alerting rule template is generic in this case and the user supplies the fields to alert on, perhaps also the connectors they want to deploy

Yes, for the connector I was expecting the user to fill that in as it's hard to predict what they want. For the fields to alert on and so on it depends on the pre-packaged alert, but it could definitely be a partial form with the user still adding stuff.

elastic / package-spec

Add alerts to integration packages #121