Repetetive Patterns in Recipes are potential sources of error. Meta Steps?

mediacloud / sous-chef

Configurable Data Analytics Pipeline

1 stars 0 forks source link

Repetetive Patterns in Recipes are potential sources of error. Meta Steps? #15

Open pgulley opened 6 months ago

pgulley commented 6 months ago

Most deployment recipes will have a pattern like "-> Sort -> Write to Disk -> Export to S3" which have a lot of shared names between them. We should have a meta step that just encapsulates all of these steps together.