Step Functions

[GenesisChris]

Is your feature request related to a problem? Please describe. Integrating Step Functions is not currently supported, but is something I find myself using

Describe the solution you'd like I would love it if Step Functions were to become officially supported to be provisioned and given access in the Amplify CLI.

Describe alternatives you've considered I'm currently manually integrating the CloudFormation behind the scenes, or alternately using Terraform as a completely separate side application which then gets called from Amplify Apps.

Additional context AWS Step Functions is a very powerful solution for a lot of backend processes that I have behind the apps I write. To be able to natively use it in Amplify would be such a big time saver. Looks like the CloudFormation itself for it is pretty easy, but using Lambdas in the state machine is something I'm still figuring out and I'm guessing is the more difficult implementation blocker to overcome.

[Ashish-Nanda]

Transferring this feature request to the amplify-cli repo since it is a CLI feature.

[rfpedrosa]

I also would like to have this as well (which is something supported on the serverless framework as a comparison). By now, we are using custom CF stacks to setup step functions

[andrestone]

Is there any work being done on this? Worth submitting ideas / PR?

[kaustavghosh06]

@andrestone Currently there isn’t work going on this. Would be great if you could submit a public RFC before working on a PR and get some comments from the core team and the community.

[mormsbee]

+1 for this. Need step functions for use with Amplify CLI.

[undefobj]

Can you give us information on your business use case? This will help us with future designs. For example "In my app I need to provide a checkout system, etc." Thanks

[mormsbee]

CRM Application. Someone creates an issue, or is even just working an issue and changes a field on a Case... either scenario could kickoff a workflow. Emails might need to be sent, tasks created and assigned, application wide notifications, the sky is the limit. At this point we are just evaluating the product but it seems like it might be a good fit. So far we've been able to stick to using the amplify cli for everything and I was actually surprised to see that there was no support for step functions currently.

[hisham]

I'm also interested in Step functions.

My workflow is essentially request comes in -> identity check needs to be done on customer -> request assigned to appropriate person to fulfill -> request fulfilled -> "deliverable" shipped digitally or physically as appropriate -> communicate to customer -> charge customer card -> after a week send NPS score request for feedback.

It's similar to e-commerce workflow: order comes in -> fraud check -> assign to appropriate fulfillment center -> ship the item -> charge customer.

[cschow-ucsd]

Can someone please share their workarounds to integrate step functions into an Amplify application? I just started learning about Amplify and would love to have a detailed guide on how that's done.

[rfpedrosa]

@cschow-ucsd I make step-functions working by creating a "Custom CloudFormation stacks". Similar to this this but for using Step Functions as orchestration instead of SNS topics for monitoring

[cristhian-boc]

@cschow-ucsd I make step-functions working by creating a "Custom CloudFormation stacks". Similar to this this but for using Step Functions as orchestration instead of SNS topics for monitoring

Thanks! I have it working following the docs and a couple of examples at Github: https://github.com/aws-samples/aws-etl-orchestrator/blob/master/cloudformation/step-functions-resources.yaml https://github.com/widdix/aws-step-functions-example/blob/master/template.yml

I'm curious when this would be released. There are some plans on supporting this, right? It was a pain in the... fingers... to make it work doing it manually (it cost me 3 complete days to do this without previous CloudFormation knowledge).

My use case is to schedule some processes to be run a couple of minutes after, to check some flags, and then act in consequence to that. I've done that using EventBridge rules but it has some absurd limitations that cannot scale well.

I think we can write the state machine definition ourselves when doing the generation of the templates, but it would be great to have the rest automated as is in the other implementations (common things like role generation, environments, outputs, parameters, etc).

[asmajlovicmars]

+1 for Step functions with Amplify, as well as SNS and SQS! Currently, I'm using custom CF for adding SNS, and SQS, and will do the same for a workflow, but would be natural to be able to add them all through CLI.

[undefobj]

Hi for to the recent commenters on this thread - could you give more details on your use case and the application you're trying to build? We're interested in the specific use case as it pertains to a mobile or web application as this will help use scope design in the future. We don't have a timeline but this will help investigate.

[asmajlovicmars]

Hi for to the recent commenters on this thread - could you give more details on your use case and the application you're trying to build? We're interested in the specific use case as it pertains to a mobile or web application as this will help use scope design in the future. We don't have a timeline but this will help investigate.

Hi @undefobj , In our case, the particular application is like a multi-sided platform, that schedules events (interviews) between candidates and organizations. During that process a workflow is supposed to move candidates' applications through different stages. In that process the system will send SMS messages, SNS notifications, update status and move applicants through different stages as the requirements are completed, or not.

The application will be served on different devices, but I wouldn't say that it's predominantly a mobile app that we're building.

Not sure if this gives enough information, and the idea about the type of application we're building?

In the past I was part of the team that built a mortgage advancing application, that used FileNet (later IBM) Visual Workflow, which in a similar fashion moved objects through a workflow based on multitude of conditions. So, I'm looking for a similar usage.

[koweblomke]

In our case we use a state machine for AWS Account creation as part of an Account Vending Machine. We are working on an Angular frontend for this Account Vending Machine with Amplify.

I ended up using the AWS API Gateway calling the step functions API. I have build a custom REST API with amplify "Custom CloudFormation stacks" which calls the step functions API.

Building this custom REST API was a bit cumbersome as the Amplify documentation is not that expressive about how to do that. The most difficult part was to get de api endpoint automatically in the aws-exports.js file when I preform an amplify push .

I eventually added a rest api with the amplify cli amplify add api and replaced the generated cloudformation code with my custom cloudformation code. This way when I perform an amplify push the endpoint is automatically added to the "aws_cloud_logic_custom" entry in the aws-exports file. I don't think this is the intended way to go but it works and I haven't found another way yet.

Perhaps it's an idea that on adding a REST API with the amplify cli you could also choose to integrate with other AWS Services like Step Functions.

[mansdahlstrom1]

+1 for step functions with amplify. Is there anyway to see if this feature is currently being looked at, or if it's included in a roadmap / feature update?

EDIT: Just saw @undefobj comment, our use case is to be able to schedule events. We are currently using Cloudwatch Event Rules but this is not very scaleable since it's limited to a quite low number of rules. (100 i think)

[idanlo]

+1

[shrishty]

+1.. I need to build an order management system.

[michaelzxu]

+1

[jimjoes]

+1 We have two use-cases:

1) Synthetic "big" data generator 2) Custom glue orchestration (in fact, being able to describe glue config via Amplify would also be awesome)

[Mayank611]

+1 Any update on this feature? It will be really useful to have step functions integration with amplify.

[AustinZhu]

+1

[chrisl777]

Use case for us with Step Functions (and SNS/SQS).

When a user visits the site and signs up, we want to send them a welcome email or text, along with a reminder email or text with some further prompts after a certain amount of time if they haven't further engaged with the site. For the case where we want to send a reminder, Step Functions are very handy, b/c they can be used as a kind of "cron" to wait a given amount of time before processing further, i.e. before kicking off the process that sends a message. The SQS parts of the flow are useful for storing large amounts of messages, so that the process will scale.

The flow generally looks like: User sign-up -> DynamoDB Stream -> Lambda -> SQS -> Lambda -> Step Functions -> SQS -> Lambda -> SNS -> Email/Text to user

Currently, we have this workaround where set up and maintain this stack using the Serverless framework, and it's a pain to manage this stack in parallel to the front-end/API which are in Amplify. But we do use some of the output of the Amplify deployment (such as the DynamoDB stream ID) as inputs for the Serverless stack, which is brittle, especially when other devs want to stand up their own personal version of the stack. Ideally, we would want this stack to be a part of our Amplify CI/CD. Also, it is hard to test this stack in conjunction with front-end work. Admittedly, we are also kicking off the deploy of this stack manually, which is a pain, we just haven't had the time or resources to invest in a better way. So if there was a straightforward way to spin up this stack, it would be greatly preferred.

[blairtaylor]

+1

[OElesin]

+1

Use case for Step Functions

We are trying to make machine learning as easy as possible for people using SQL through Amazon Athena. We want to provide a unified experience for data analysts to leverage SageMaker from our SQL Interface.

User flow: User writes a SQL query with the target column and then submits a model training job. SQL Query + Target Column -> Step Function -> [create athena query execution -> write query to S3 -> Start SageMaker Autopilot job -> package model -> deploy endpoint] -> user makes predictions with Athena ML.

So far, we have not built anything yet but our assumption is that a Lambda function can trigger our state machine.

[mikerudge]

+1

[jonmifsud]

This would be something we'd love to have.

Heavy users of both Amplify and Step Functions - not as much in combination, unfortunately - as we always have to set up the Step Functions manually and then make sure we have the right permissions on the Lambdas.

[two-pack]

+1

[nniirraavv]

+1 for step functions with Amplify

[nijatb]

+1

[jeanbaptistepriez]

+1

[alexhouse]

Would be great to see this included. I have a bit of a hybrid system where one Dynamo record can be updated by two different sources. Step functionality would allow me to only run certain lambda functions depending on the originating source (API Gateway triggers a lambda function to update a record; Dynamo Stream then fires another lambda function that sends an API request to a third party)

[MorpheusNephew]

+1

[bcs-gbl]

+2

[413n]

+1

[PederAaby]

+1

[johnf]

To add an extra use case I'm using Wait in Step functions to be able to trigger lambda functions in the future. For example, I need to send a push notification to a user in 2 days to remind them of something.

[hisham]

@johnf one option for your use case is to use dynamo table with TTL property and stream it to a lambda. See https://theburningmonk.com/2019/03/dynamodb-ttl-as-an-ad-hoc-scheduling-mechanism/

[OperationalFallacy]

Given AppSync limit for queries at 30 sec, there are no reliable alternatives to Step Functions for long-running queries or mutations.

https://dev.to/aws-builders/how-to-call-aws-step-functions-from-aws-amplify-21lm - great example how to implement with CDK

There is a fair amount of VTL pain, if Amplify team could take care of it and have SFN native integration - that would be Amazing addition.

[bcs-gbl]

Given AppSync limit for queries at 30 sec, there are no reliable alternatives to Step Functions for long-running queries or mutations.

You can use AWS Batch job for long running tasks.