[Security Solution] Instrument Security frontend code to collect performance data

[xcrzx]

Epic: https://github.com/elastic/kibana/issues/130971

Overview

Currently, with enabled APM integration, we collect too little data to assess the performance of Security Solution. Here are a couple of examples:

Route change transactions

User interaction transactions

On those screens, it's hard to find the information I'm currently interested in. For example, if I work on a task affecting the Rule Management page, it would be helpful to isolate only that page and investigate possible performance issues in isolation.

We could improve frontend code instrumentation to visualize code paths similar to what we've done to rule executors: https://github.com/elastic/kibana/pull/117672. Custom frontend spans or transactions we can add to measure performance indicators:

• The time needed to display data on a page. E.g. how much time does it take for the rules management page to become fully interactable?
• The time needed to perform user-critical tasks. Examples of such tasks are:
  • Rules filtering and sorting
  • Bulk actions application
  • Create new rules
  • Enable/disable rules
  • etc.
• Initial page load time and Time to Interactive (TTI)
• Number of HTTP requests and the amount of data transferred for a single user interaction

Other opportunities

• Implement automatic report generation using Lighthouse.
• Use unit tests with a sufficient amount of input data to automate monitoring of the performance of critical algorithms.
• Use React Profiler to measure the performance of frontend components.

Next steps

• Introduce data sets for local development with enough data for performance bottlenecks to start to show up
• Enable APM instrumentation for local Cypress tests
• Explore possibilities of automated performance regression testing
• Enable APM data collection for internal clusters + set up alerts to get notifications when performance falls below a certain threshold

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[oatkiller]

In order to fully understand the performance issues being suffered by Customer Zero, and to develop an automated way to detect regressions, I would like to consider including the instrumentation of sourcerer APIs as part of this work. Let's talk more.

[lizozom]

Where is this data from? An internal collection cluster?

[xcrzx]

Where is this data from? An internal collection cluster?

@lizozom This is from my local Kibana. You can enable APM integration by setting elastic.apm.active: true in Kibana config to send performance metrics to any APM server you want.