Failing test: Rules Management - Prebuilt Rules Management Integration Tests - Serverless Env - Complete License.x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/management/trial_license_complete_tier/get_prebuilt_rules_status·ts - Rules Management - Prebuilt Rules - Prebuilt Rules Management @ess @serverless @skipInServerlessMKI Prebuilt Rules status get_prebuilt_rules_status should return empty structure when no prebuilt rule assets

[kibanamachine]

A test failed on a tracked branch

JestAssertionError: expect(received).toMatchObject(expected)

- Expected  - 2
+ Received  + 2

  Object {
    "num_prebuilt_rules_installed": 0,
-   "num_prebuilt_rules_to_install": 0,
+   "num_prebuilt_rules_to_install": 216,
    "num_prebuilt_rules_to_upgrade": 0,
-   "num_prebuilt_rules_total_in_package": 0,
+   "num_prebuilt_rules_total_in_package": 216,
  }
    at Context.<anonymous> (get_prebuilt_rules_status.ts:42:23)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:73:16) {
  matcherResult: {
    message: '\x1B[2mexpect(\x1B[22m\x1B[31mreceived\x1B[39m\x1B[2m).\x1B[22mtoMatchObject\x1B[2m(\x1B[22m\x1B[32mexpected\x1B[39m\x1B[2m)\x1B[22m\n' +
      '\n' +
      '\x1B[32m- Expected  - 2\x1B[39m\n' +
      '\x1B[31m+ Received  + 2\x1B[39m\n' +
      '\n' +
      '\x1B[2m  Object {\x1B[22m\n' +
      '\x1B[2m    "num_prebuilt_rules_installed": 0,\x1B[22m\n' +
      '\x1B[32m-   "num_prebuilt_rules_to_install": 0,\x1B[39m\n' +
      '\x1B[31m+   "num_prebuilt_rules_to_install": 216,\x1B[39m\n' +
      '\x1B[2m    "num_prebuilt_rules_to_upgrade": 0,\x1B[22m\n' +
      '\x1B[32m-   "num_prebuilt_rules_total_in_package": 0,\x1B[39m\n' +
      '\x1B[31m+   "num_prebuilt_rules_total_in_package": 216,\x1B[39m\n' +
      '\x1B[2m  }\x1B[22m',
    pass: false
  }
}

First failure: kibana-on-merge - main

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[banderror]

This test failure and https://github.com/elastic/kibana/issues/190952 are failures of the same test get_prebuilt_rules_status should return empty structure when no prebuilt rule assets in two different builds and in both ESS and Serverless.

Both test runs were retried and succeeded, but nonetheless it strongly feels as flakiness of the test itself. The test expects to have 0 prebuilt rule assets in the index but there are clearly some of them, which might indicate an issue with test isolation.

I'll skip the test if it continues to fail and our team will work on fixing it next week.

[kibanamachine]

New failure: kibana-on-merge - main

[delanni]

@banderror - we're getting anomaly warnings about this failing on several PR builds also, so it's confirmed to be flaky. I suggest skipping this as it may well be blocking devs

[kibanamachine]

New failure: kibana-on-merge - main

[kibanamachine]

New failure: kibana-on-merge - main

[delanni]

/skip

[kibanamachine]

Skipped

main: c4a39e1291dc3c8e23db0104a8270c0b39324fd1

[banderror]

@delanni Yes, now it's clear that it's a flaky test. Thanks for skipping.