elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.69k stars 8.12k forks source link

[Security Solution] Fix and unskip tests imports API integration tests #188673

Open nikitaindik opened 1 month ago

nikitaindik commented 1 month ago

PR https://github.com/elastic/kibana/pull/188291 fixed a bug which prevented some of our tests from running in Serverless PR pipelines. With the fix in place, it turned out that these tests are failing and need to be updated.

AC:

elasticmachine commented 1 month ago

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine commented 1 month ago

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine commented 1 month ago

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

xcrzx commented 1 month ago

The perform_bulk_action_dry_run.ts failure is related to the manual rule runs feature. It is being disabled in the serverless environment:

[
  {
    message: 'Manual rule run feature is disabled.',
    status_code: 500,
    err_code: 'MANUAL_RULE_RUN_FEATURE',
    rules: [ [Object], [Object] ]
  }
]

We need someone from @elastic/security-detection-engine to help with that.

e40pud commented 1 month ago

Manual rule run feature is hidden behind feature flag and feature flags are not support on MKI. We should mark related tests with @skipInServerlessMKI, something like describe('@skipInServerlessMKI schedule manual rule run action', ...); in perform_bulk_action_dry_run.ts.

xcrzx commented 1 month ago

The other issue I tracked down is related to the migrate pre-8.0 action connector ids test suite:

https://github.com/elastic/kibana/blob/f1eae5c92449332313ceb32189915a3fc5835921/x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/rule_import_export/trial_license_complete_tier/import_rules.ts#L924

Somewhere during that test execution, saved object migrations fail badly with the following error:

proc [kibana] [2024-07-22T14:24:33.804+02:00][ERROR][http] 500 Server Error {"http":{"response":{"status_code":500},"request":{"method":"post","path":"/internal/saved_objects/_migrate"}},"error":{"message":"Unable to complete saved object migrations for the [.kibana] index: Index .kibana_1 is using an incompatible version of the v2 algorithm"},"service":{"node":{"roles":["background_tasks","ui"]}}}

After the migration fails, all security solution APIs start responding with authentication errors, such as Unauthorized to find rules for any rule types. It seems all roles lose their capabilities at that stage.

dhurley14 commented 1 month ago

@xcrzx Thanks for looking into this. We believe we can skip these action saved object migration tests in serverless since we do not expect a pre-8.0 stack to exist in serverless.

szwarckonrad commented 1 month ago

https://github.com/elastic/kibana/pull/188291 was merged :)