elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.57k stars 8.21k forks source link

[Custom threshold] Add span label for async operations during rule execution #186503

Open maryam-saeidi opened 4 months ago

maryam-saeidi commented 4 months ago

Summary

In order to be able to have a better overview of how long different async operations take, we want to add better span labeling around async operations during the custom threshold rule execution. As an example, you can check how the security team does this using withSecuritySpan.

const timestampFieldCaps = await withSecuritySpan('fieldCaps', () =>
  services.scopedClusterClient.asCurrentUser.fieldCaps(
    {
      index: inputIndex,
      fields: secondaryTimestamp
        ? [primaryTimestamp, secondaryTimestamp]
        : [primaryTimestamp],
      include_unmapped: true,
      runtime_mappings: runtimeMappings,
      ignore_unavailable: true,
    },
    { meta: true }
  )
);

There is an ongoing effort to label spans in the alerting framework as well.

Note

We only consider adding span for async operations, although we can do that for sync operations that we think might take long, but keep in mind that adding many spans will have a negative impact on performance and possibly debugging.

Implementation related info

Add the following to your Kibana config:

elastic.apm.active: true
elastic.apm.transactionSampleRate: 1.0
elastic.apm.environment: YOUR_USERNAME

Going to https://kibana-cloud-apm.elastic.dev/ -> APM page, you should be able to filter the environment using your YOUR_USERNAME.

Acceptance criteria

elasticmachine commented 4 months ago

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)