[Watcher] Watch execution history shows OK when condition is met, but throttle in effect

[alisonelizabeth]

Summary

The watch execution history UI shows an OK state when the condition is met, but throttling is still in effect.

Steps to reproduce:

1. Create a new watch that is expected to fire. Specify throttle_period_in_millis (docs).

2. Navigate to the watch execution history page.

   • The watch execution history should show an execution with the Firing state
   • The next execution should show a Firing state, but also show Throttled in the Comment column.

3. Prevent the watch from firing.

   • The next execution should show an OK state.

4. Trigger the watch condition again within the throttle period.

   • The next execution still shows an OK state.

Expected behavior: The last execution from step 4 should have a state equal to Firing and also show Throttled in the Comment column.

---

Watch status data from step 2(b):

  "status": {
    "state": {
      "active": true,
      "timestamp": "2020-03-04T15:01:44.616Z"
    },
    "last_checked": "2020-03-04T15:05:26.835Z",
    "last_met_condition": "2020-03-04T15:05:26.835Z",
    "actions": {
      "send_email": {
        "ack": {
          "timestamp": "2020-03-04T15:04:57.326Z",
          "state": "ackable"
        },
        "last_execution": {
          "timestamp": "2020-03-04T15:04:57.326Z",
          "successful": true
        },
        "last_successful_execution": {
          "timestamp": "2020-03-04T15:04:57.326Z",
          "successful": true
        },
        "last_throttle": {
          "timestamp": "2020-03-04T15:05:26.835Z",
          "reason": "throttling interval is set to [5m] but time elapsed since last execution is [29.5s]"
        }
      }
    },
    "execution_state": "throttled",
    "version": -1
  }

Watch status data from step 4

  "status": {
    "state": {
      "active": true,
      "timestamp": "2020-03-04T15:01:44.616Z"
    },
    "last_checked": "2020-03-04T15:08:26.897Z",
    "last_met_condition": "2020-03-04T15:08:26.897Z",
    "actions": {
      "send_email": {
        "ack": {
          "timestamp": "2020-03-04T15:06:56.866Z",
          "state": "awaits_successful_execution"
        },
        "last_execution": {
          "timestamp": "2020-03-04T15:04:57.326Z",
          "successful": true
        },
        "last_successful_execution": {
          "timestamp": "2020-03-04T15:04:57.326Z",
          "successful": true
        },
        "last_throttle": {
          "timestamp": "2020-03-04T15:08:26.897Z",
          "reason": "throttling interval is set to [5m] but time elapsed since last execution is [3.5m]"
        }
      }
    },
    "execution_state": "throttled",
    "version": -1
  }

It looks like this is occurring because we are returning an OK state whenever the ack state equals awaits_successful_execution (code). We probably also need to check the execution_state to see if it is throttled.

[elasticmachine]

Pinging @elastic/es-ui (Team:Elasticsearch UI)

[elasticmachine]

Pinging @elastic/kibana-management (Team:Kibana Management)

[alisonelizabeth]

This needs to be triaged again to see if it's still valid. There were some improvements made to the way we display the watch status in https://github.com/elastic/kibana/pull/138563.