I'm trying to iterate over a set of partitions using Glue's GetPartitions call. The first response may or may not have a token to continue pagination. My workflow looks like this:
Call GetPartitions
Map State
Do mapped actions in parallel
Check if the token is in state input
If so, call GetPartitions with NextToken parameter
If not, end the state machine
This is completely possible in Cloudformation, states language, and Workflow Studio. However, the SDK seems to make this impossible. I cannot put the same State into the workflow twice since it validates that the state ID is unique. I also cannot set the Next Step on the subsequent GetPartitions call since the Task class doesn't allow it. I tried to override it manually but that seemed to break the validator as it threw a "missing state ID" type error.
Use Case
This feature is necessary to a lot of stepfunctions-native development. One of the best parts of stepfunctions is integrations and many of them are useless without this. Essentially, without this, the SDK is limited to acyclic graphs only which is a major loss.
Proposed Solution
There are multiple solutions.
Task class could support a next step
Chain class could interpret duplicate states as "next states"
Drop-in states language in general could be more well-supported
Overriding fields like next_step on Task instances could just work instead of raising an inaccurate exception
I'm trying to iterate over a set of partitions using Glue's GetPartitions call. The first response may or may not have a token to continue pagination. My workflow looks like this:
This is completely possible in Cloudformation, states language, and Workflow Studio. However, the SDK seems to make this impossible. I cannot put the same State into the workflow twice since it validates that the state ID is unique. I also cannot set the Next Step on the subsequent GetPartitions call since the Task class doesn't allow it. I tried to override it manually but that seemed to break the validator as it threw a "missing state ID" type error.
Use Case
This feature is necessary to a lot of stepfunctions-native development. One of the best parts of stepfunctions is integrations and many of them are useless without this. Essentially, without this, the SDK is limited to acyclic graphs only which is a major loss.
Proposed Solution
There are multiple solutions.
This is a :rocket: Feature Request