AWS Step Functions is an orchestration service for reliably executing multi-step processes using visual workflows. This repository includes detailed examples that will help you unlock the power of serverless workflow.
MIT No Attribution
225
stars
83
forks
source link
Demo using Distributed Map to process NOAA data set #34
This is an example of how to use the Distrubted Map feature of Step Functions to perform a map/reduce analysis of numeric data.
This example also used Distributed Map with standard state input/output, with a nested inline map, to copy data from the public NOAA data set to your own bucket.
How
Note, this must be run in us-east-1
Created a SAM application that creates three workflows that: - copy S3 data to your bucket - performs the analysis with Distributed Map - deletes S3 data in two buckets
Added a description and instructions to run in your own account in the README
Added instructions on how to clean up the deployed resources
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Why
How
us-east-1By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.