Expiration policy for signal requestIDs in mutable state

[dhiaayachi]

Is your feature request related to a problem? Please describe. Today if workflow is signal directly via the Signal api (not workflow-to-workflow signal), signal requestID will be store in workflow mutable state forever, which will result in an increase in mutable state size, which could lead to performance issue when say there're 30K signal requestIDs.

Describe the solution you'd like

• Simply rely on the existing max signal count limit.
• Give up the existing guarantee we have on signal deduplication and remove those requestIDs before workflow is closed.
  • Expire requestID based on time (still need to enforce the max signal count limit)
  • Expire requestID based on count

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

[dhiaayachi]

Thank you for reporting this.

This is an interesting issue with Temporal's workflow signal mechanism. It seems the current design might lead to growing workflow mutable state sizes when handling external signals. You are suggesting ways to mitigate this.

The provided documentations do not have specific workarounds for this particular issue, however, they contain valuable insights into Temporal's workflow message passing mechanisms, specifically focusing on the differences between Signals, Queries, and Updates.

For a possible workaround, you might consider:

• Using a custom signal handling logic within your workflow: Instead of storing every signal's request ID, you could implement a counter within your workflow that tracks the number of signals received. This counter could be used to manage signals in a more memory-efficient way.

To further understand the nature of the issue, I would need some more information about your workflow implementation.

• Could you share your current workflow implementation?
• What are the specific types of signals that your workflow receives?
• What kind of performance issues are you observing?

I'm happy to help further as you provide more context!

[dhiaayachi]

Thank you for reporting this issue. I understand that storing signal request IDs in workflow mutable state forever can lead to performance issues, especially when dealing with a large number of signals.

Currently, there is no built-in mechanism to automatically expire or remove signal request IDs from workflow mutable state. The closest option that you have already mentioned is using the existing max signal count limit. This limit ensures that the number of signals that a workflow can receive is capped, preventing an excessive buildup of request IDs in mutable state.

We appreciate your feedback and will consider your feature request for future releases.