elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.47k stars 8.04k forks source link

Add a circuit breaker to the size of the state object in Task Manager tasks #182186

Open mikecote opened 2 months ago

mikecote commented 2 months ago

Problem Statement

We have no guardrails when it comes to storing state within a Task Manager task, allowing some implementations to abuse what is stored within the state, causing the Task Manager performance to degrade by serializing / deserializing large amounts of data during the claiming phase.

We should start by logging a warning when we save a task with a state larger than 1 MB. This will let us collect how frequently this is happening.

Definition of Done

elasticmachine commented 2 months ago

Pinging @elastic/response-ops (Team:ResponseOps)