Discussion about Offline execution

[ChristopheBougere]

This is a Feature Proposal

Description

What should be solved

Ability to execute a state machine locally. This would be really helpful to ease development. I'd like to open discussion on allowing resources creation using AWS StepFunctions API on a specific endpoint when an option is passed.

How can we achieve it

Using stepfunctions-local, anyone can start a local stepfunctions server which is AWS-API compliant (goal being to be as close as possible from the AWS engine, like with serverless-dynamodb-local and serverless-s3-local which are using local server with the same API).

Then we could either add an --offline/--local option to serverless invoke:

sls invoke stepf --name <stepfunctionname> --data '{"foo":"bar"}' --offline

or create a new subcommand invoke local (which would look like the official serverless invoke local for lambda):

sls invoke local stepf --name <stepfunctionname> --data '{"foo":"bar"}'

Instead of deploying resources to CloudFormation, this command would parse the stepfunction definition in serverless.yml and create resources using AWS StepFunctions API on the defined endpoint (like http://localhost:4584). Basically, we don't need IAM roles and we would have to:

• Create the state machine
• Create the potential activities
• Start an execution
• Wait until the execution is finished and display the output or the error. => [To be discussed] it could also simply return the executionArn

Additional config

• Stepfunctions server endpoint (hostname + port) and region

Similar or dependent PR:

• 90 was only adding the validation, which may not be necessary anymore with a way to execute state machines locally

Let me know your opinion on this, I would be happy to submit a PR if you're interested in this feature.

[horike37]

Hi @ChristopheBougere, thank you for opening this issue :+1: Recently, serverless-step-functions-offline has been released for local development. How about use that?

[ChristopheBougere]

Hi @horike37 Here are some concerns I have with serverless-step-functions-offline. I think it can be really useful for a simple use case, but could be blocking on more complicated workflows.

• It does not support activities
• It is limited to Node.js
• It plays the role of lambda emulator while in my opinion it should be handled by a specific program (localstack, or serverless-offline) which already are handling lots of cases specific to lambda context/runtimes/...
• It is serverless dependant. My point of view is that the local engine shouldn't be related to a framework, it should be integrated in it
• We can't use StepFunctions CLI or SDK

[theburningmonk]

@ChristopheBougere AWS has also published a tool themselves for running Step Functions locally

[hotgazpacho]

While interesting, if you want an offline experience for your step functions, you still need to provide emulated services for, at a bare minimum, Lambda (but probably also DynamoDB, SQS, SNS, etc.).

Does anyone have a working setup for that?

Is LocalStack the answer for providing simulacra for those other services?

[jkruse14]

I have a very basic package here: https://www.npmjs.com/package/serverless-offline-step-functions

It needs work, but is a good starting point and any feedback would be great

[pianomansam]

I just started on a plugin that supports step functions in serverless offline.

https://github.com/pianomansam/serverless-offline-stepfunctions

I also created a plugin to automatically start/stop the wonderful stepfunctions-local by @tristanbenier and @ChristopheBougere so we have a local Step Functions server:

https://github.com/pianomansam/serverless-offline-stepfunctions-local