search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.65k stars 8.23k forks source link

[ResponseOps][Task Manager][mget Claimer] add "stale" counter to task manager metrics #184943

Open pmuellr opened 5 months ago

pmuellr commented 5 months ago

In PR implement task claiming strategy mget #180485 we implemented an alternative task claiming strategy, but it has the following problem:

We have a new thing we'd like to count for task manager metrics - "stale" documents. These are documents which were returned as candidates from the search, but for which the mget indicates the document is stale. This is similar to a conflict, but distinct, since we also can get conflicts when we bulk update the tasks to mark them claimed. So, we need another slot.

Kinda more meta, I found the existing task manager metrics fiddly to deal with. I'd love to be able to have docs written to the event log, for instance, which would be easy to slice and dice in Kibana Discover / Dashboards. Maybe we can implement something there and make it optional by config? I don't think we ALWAYS want to be generating EL docs, but being able to do it as an opt-in could be useful.

elasticmachine commented 5 months ago

Pinging @elastic/response-ops (Team:ResponseOps)