opendistro-for-elasticsearch / alerting

📟 Open Distro Alerting Plugin
https://opendistro.github.io/for-elasticsearch/features/alerting.html
Apache License 2.0
279 stars 80 forks source link

[BUG] No setting available for Monitor max triggers #387

Open gmfmi opened 3 years ago

gmfmi commented 3 years ago

Describe the bug The number of Triggers allowed by monitors should not be hardcoded.

When adding a Monitor that contains more than 10 Triggers, this error is returned :

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "Monitors can only support up to 10 triggers."
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "Monitors can only support up to 10 triggers."
  },
  "status" : 400
}

I did not found any good reason to hardcode this value to "10". As an example, the total number of Monitor default value is set to 1000 but can be dynamically redefined.

Other plugins installed (not useful)

To Reproduce Steps to reproduce the behavior:

POST _opendistro/_alerting/monitors
{
  "type": "monitor",
  "schema_version": 3,
  "name": "Error",
  "enabled": true,
  "schedule": {
    "period": {
      "interval": 1,
      "unit": "MINUTES"
    }
  },
  "inputs": [
    {
      "search": {
        "indices": [
          "*"
        ],
        "query": {
          "size": 0,
          "query": {
            "match_all": {}
          }
        }
      }
    }
  ],
  "triggers": [
    {
      "name": "ID 1",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 2",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 3",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 4",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 5",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 6",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 7",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 8",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 9",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 10",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    },
    {
      "name": "ID 11",
      "severity": "3",
      "condition": {
        "script": {
          "source": "return false",
          "lang": "painless"
        }
      }
    }
  ]
}

Expected behavior Have a dedicated setting in the alerts section. As an example, it could look like: opendistro.alerting.monitor.max_triggers (to be consistant with the existing opendistro.alerting.monitor.max_monitors). See https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/settings/#alerting-settings.

Desktop:

Additional context I run the OpenDistro stack using Docker with image v1.13.2.

gmfmi commented 3 years ago

I take a look at the source code, here is a quick solution that should to the trick.

In the file /alerting/src/main/kotlin/com/amazon/opendistroforelasticsearch/alerting/settings/AlertingSettings.kt

Replace:

const val MONITOR_MAX_TRIGGERS = 10

By:

val MONITOR_MAX_TRIGGERS = Setting.intSetting(
    "opendistro.alerting.monitor.max_triggers",
    10,
    Setting.Property.NodeScope, Setting.Property.Dynamic
)