elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.76k stars 8.16k forks source link

[Meta] Improve observability alerting logging #193881

Open maryam-saeidi opened 5 days ago

maryam-saeidi commented 5 days ago

Summary

Currently, we use two loggers for observability rules, and in order to enable those, we need to add two configurations. In this ticket, we want to improve this logic by only using the alerting logger to remove additional loggers on our side. (https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/[…]/server/lib/rules/custom_threshold/custom_threshold_executor.ts)

This is an example of config for loggers for the custom threshold rule:

logging:
  loggers:
    - name: plugins.alerting.observability.rules.custom_threshold
      level: trace
    - name: plugins.observability.thresholdRule
      level: trace

We also need to ensure that the logs have proper tagging for easier investigation. (In case we want to add more tags besides the ones provided by the alerting framework.) In this PR @ymao1 included the rule ID and the rule type ID in the tags. This can be verified locally by enabling this config:

logging:
  appenders:
    console_appender:
      type: console
      layout:
        type: json
  root:
    appenders: [console_appender]
    level: info
elasticmachine commented 5 days ago

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)