[Security Solution] Flaky test: bulk_edit_rules_actions.cy.ts

banderror commented 6 months ago

Summary

One of our Cypress tests for bulk editing rule actions failed in the Serverless CD pipeline for Elasticsearch. The test was retried and succeeded on the 2nd try, so we haven't got an automatically created ticket.

Build: https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote/builds/565#018da2c5-8c87-4a0a-863f-794d5b38b736

Test path:

e2e/detection_response/rule_management/rule_actions/bulk_actions/bulk_edit_rules_actions.cy.ts:162:32

Test name:

Detection rules, bulk edit of rule actions
  Restricted action privileges
    User with no privileges can't add rule actions

Error:

AssertionError: Timed out retrying after 150000ms: Expected to find element: `[data-test-subj^="callout-missing-user-privileges"]`, but never found it.

Let's look into it and:

Audit the test for potentially incorrect setup/teardown in before* and after* hooks.
Run it through the flaky test runner with a large number of iterations.
Try to reproduce it locally by running it many times (Cypress._.times could be used for that).

elasticmachine commented 6 months ago

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine commented 6 months ago

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine commented 6 months ago

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

elastic / kibana

[Security Solution] Flaky test: bulk_edit_rules_actions.cy.ts #176908

Summary