If a real-time datafeed has never seen any data (including during any initial training period) then it will automatically stop itself and close its associated job after this many real-time searches that return no documents. In other words, it will stop after frequency times max_empty_searches of real-time operation. If not set then a datafeed with no end time that sees no data will remain started until it is explicitly stopped. By default this setting is not set.
The special value -1 unsets this setting.
As a result, users Security ML Jobs may automatically stop and it may not be clear to them as why this may be occurring, or how to mitigate.
This enhancement is for improving the UI/UX around this behavior, which may include:
Surfacing the log generated when the job is stopped to the UI, so users know why the job is no longer running. (e.g. directly in ML Jobs Settings UI, or via alerting/Kibana Notification center)
Provide link to docs for max_empty_searches, and potentially option to update (or instructions for how to do so via ML app).
Investigate alternate options with ML team, perhaps an additional configuration to have the job/datafeed auto-re-enabled once relevant data starts flowing again (some sorta poller?)
The Security Solution ML Jobs make use of the
max_empty_searches
configuration, which means:As a result, users Security ML Jobs may automatically stop and it may not be clear to them as why this may be occurring, or how to mitigate.
This enhancement is for improving the UI/UX around this behavior, which may include:
max_empty_searches
, and potentially option to update (or instructions for how to do so via ML app).