elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.41k stars 24.87k forks source link

[ML] Remote clusters can't be excluded using "-" from AD datafeed #114509

Open wwang500 opened 1 month ago

wwang500 commented 1 month ago

Version: 8.15.1

Step to reproduce:

  1. Setup two remote clusters: cluster1, cluster2
  2. Load kibana sample data on both remote cluster1 and remote cluster2
  3. Create a dataview to exclude one of them: ":kibana,-cluster1:*"
  4. Create a simple AD job using the above dataview
  5. at the last step of AD job wizard, it will hit a no_such_remote_cluster_exception error:
{
  "statusCode": 404,
  "error": "Not Found",
  "message": "[no_such_remote_cluster_exception\n\tRoot causes:\n\t\tno_such_remote_cluster_exception: no such remote cluster: [-cluster1]]: no such remote cluster: [-cluster1]",
  "attributes": {
    "body": {
      "error": {
        "root_cause": [
          {
            "type": "no_such_remote_cluster_exception",
            "reason": "no such remote cluster: [-cluster1]"
          }
        ],
        "type": "no_such_remote_cluster_exception",
        "reason": "no such remote cluster: [-cluster1]"
      },
      "status": 404
    }
  }
}

Workaround:

  1. don't use "-" to exclude, instead, explicitly include remotes by names
  2. use a query to exclude remote clusters:
    "bool": {
      "must_not": [
        {
          "terms": {
            "_index": [
              "cluster1:*"
            ]
          }
        }
      ]
    }
elasticsearchmachine commented 1 month ago

Pinging @elastic/ml-core (Team:ML)