elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.66k stars 8.23k forks source link

[Security Solution] Alert Migration API does not work on 8.x alerts indices #173787

Open rylnd opened 11 months ago

rylnd commented 11 months ago

Describe the bug: Our Detection Alerts Migration API is listed as a fully functional API on 8.x. However, when attempting to migrate a newer 8.x alerts index, the migration will not succeed.

Kibana/Elasticsearch Stack version: 8.x

Steps to reproduce:

  1. Initialize the Security Solution and generate some alerts
  2. Trigger a rollover of the alerts write index
  3. Create a migration on the old index

Current behavior: If the instance does not contain an index template from 7.x alerts, the following error will be returned from the create_migration API:

{
  "message": "Cannot migrate due to the signals template being out of date. Latest version: [77], template version: [0]. Please visit Detections to automatically update your template, then try again.",
  "status_code": 400
}

If the instance does contain a 7.x index template, then creation of the migration is allowed, but finalization will fail due to the reindex not completing successfully:

The source and destination indexes have different document counts. Source [.internal.alerts-security.alerts-default-000001] has [239750] documents, while destination [.internal.alerts-security.alerts-default-000001-r000077] has [0] documents. 

Expected behavior: The Create Migration API should reject an attempt to migrate newer indices, and the Migration Status API should not show newer indices as "outdated"

elasticmachine commented 11 months ago

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine commented 8 months ago

Pinging @elastic/security-detection-engine (Team:Detection Engine)

yctercero commented 1 month ago

@rylnd we'll catch up on this, but I think we may be looking to remove these endpoints for next major. As part of cleanup, I think it's important we fix this for 8.x.