[BUG] Security Analytics Detector creation overwrites Data-Stream template Index-Patterns in OpenSearch

[rafaelma]

What is the bug?

When creating a Security Analytics Detector with a data-stream "Data source" in OpenSearch, the index-patterns defined in the data-stream template is being overwritten. The creation of the detector modifies the existing index pattern to match the specific data stream used by the detector and adds a new link to a .opensearch-sap-alias-mappings-component... component_template. This action renders the template unusable for other data streams that were designed to match the original index pattern.

How can one reproduce the bug?

Steps to reproduce the behavior:

1. Define a data-stream template with an index pattern, e.g., "ss4o_logs-dns-bind9.querylog*".
2. Define a data stream named"ss4o_logs-dns_bind9.querylog-prod" to use the aforementioned template.
3. Define another data stream named "ss4o_logs-dns_bind9.querylog-test" that uses the same template.
4. Create a Security Analytics Detector for the production environment, with "Data Source" set to "ss4o_logs-dns_bind9.querylog-prod".
5. Inspect the data-stream template after the detector creation.
6. Notice that the original index-pattern"ss4o_logs-dns-bind9.querylog*" has been changed to"ss4o_logs-dns-bind9.querylog-prod*".
7. Additionally, identify that the template has acquired a new link to a component template named ".opensearch-sap-alias-mappings-component-ss4o_logs-dns_bind9.querylog-prod".
8. Realize that the data-stream template is no longer valid for "ss4o_logs-dns_bind9.querylog-test" due to the index pattern modification.

What is the expected behavior?

Creating a Security Analytics Detector should link a new component template to the template used by the "Data Source" without modifying the existing index pattern in the template. This would allow the template to remain valid for all data streams matching the original index pattern design.

Actual Result: The creation of the Security Analytics Detector causes an alteration in the data-stream template index pattern, from "ss4o_logs-dns-bind9.querylog*" in our example to a more specific pattern matching the production data stream. In effect, this breaks the template for any other data stream that was supposed to use the same pattern, such as "ss4o_logs-dns_bind9.querylog-test".

What is your host/environment?

• OS: Red Hat Enterprise Linux release 9.4
• Version Opensearch 2.14
• Plugins

  # /usr/share/opensearch/bin/opensearch-plugin list
  opensearch-alerting
  opensearch-anomaly-detection
  opensearch-asynchronous-search
  opensearch-cross-cluster-replication
  opensearch-custom-codecs
  opensearch-flow-framework
  opensearch-geospatial
  opensearch-index-management
  opensearch-job-scheduler
  opensearch-knn
  opensearch-ml
  opensearch-neural-search
  opensearch-notifications
  opensearch-notifications-core
  opensearch-observability
  opensearch-performance-analyzer
  opensearch-reports-scheduler
  opensearch-security
  opensearch-security-analytics
  opensearch-skills
  opensearch-sql

Do you have any screenshots? If applicable, add screenshots to help explain your problem.

Do you have any additional context?

This issue occurs only if the .opensearch-sap-alias-mappings-component* component template doesn't exist, and its reference is not already included in the data-stream template. The system should ensure that any new configurations added to cater to the detector do not negatively impact the underlying template structures and their reusability.

The code responsible for this behavior is probably:

• https://github.com/opensearch-project/security-analytics/blob/7f5d2c5e07755d95f3bf74130b46f1e254096155/src/main/java/org/opensearch/securityanalytics/mapper/IndexTemplateManager.java#L184
• https://github.com/opensearch-project/security-analytics/blob/7f5d2c5e07755d95f3bf74130b46f1e254096155/src/main/java/org/opensearch/securityanalytics/mapper/IndexTemplateManager.java#L219

[dblock]

[Catch All Triage - 1, 2, 3]

[cheapshot2000]

Any thoughts on how to mitigate this? We ran into same issue just not with data stream indexes templates. We had simple index template, created a detector, and the detector then changed the index pattern within the template. That rendered the new indexes to not be found/created using the right template and the effects domino'ed from there.