apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.74k stars 4.21k forks source link

[Feature Request]: Pipeline Cost Estimation Tool #27271

Open svetakvsundhar opened 1 year ago

svetakvsundhar commented 1 year ago

What would you like to happen?

Create a tool that generates sample data and estimates costs of pipelines when being run. This tool can then be used by Beam users to predict the cost of their pipeline based on pipeline config.

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

svetakvsundhar commented 1 year ago

cc: @pranavbhandari24