search

temporalio / temporal

Temporal service
https://docs.temporal.io
MIT License
11.51k stars 823 forks source link

List Workflow Executions ignoring previous runs #351

Open mfateev opened 4 years ago

mfateev commented 4 years ago

From Slack:

We’re considering using Cadence to run something akin to the actor model. We will have an actor for every “user”, of which there are tens of millions, and each actor will be in charge of processing messages for its corresponding user. We would love to be able to query cadence for all actors that are currently in a failed state. This is a payments related use case, and its important for us to know every actor that is currently stuck. With that, we want to filter out all actors that are no longer in a failed state. That is, if there was an actor that had a workflow execution fail, but has since had a workflow execution succeed, we consider that actor fixed, and we want it filtered out of the query. Is there any way to do this? If I understand correctly, a lot of the listing features that Cadence provides list workflow executions, but what I really want is to list the set of most recent executions per workflow.

barakalon commented 4 years ago

@mfateev any idea of the difficulty/priority of this feature?

Also, is this possible to achieve today with elasticsearch/search attributes enabled? Looking at docs, I don't see that it is.

alexshtin commented 3 years ago

How do you want to get this data/run this query? Is issuing a direct query to Elasticsearch an option? It definitely not supported with Temporal list APIs (both tctl and SDKs) but most likely supported with Elasticsearch aggregations.

barakalon commented 3 years ago

Ideally, this query can be run directly from the UI/tctl.

As for issuing queries directly against Elasticsearch- that would be a big compromise for us. We want to use this query for manual operations, so its important to have some kind of user interface here. We'd like to avoid building some new CLI for this purpose or having to spin up something like Kibana. Especially since Elasticsearch - IIUC - is an internal component of the Temporal service, and it might require us depending on undocumented, unstable interfaces.

yiminc commented 1 year ago

Looks like we need additional index on current run.