search

DataCater / datacater

The developer-friendly ETL platform for transforming data in real-time. Based on Apache Kafka® and Kubernetes®.
https://datacater.io
Other
82 stars 4 forks source link

Come up with reasonable default resource requests for pipelines #20

Closed ChrisRousey closed 1 year ago

ChrisRousey commented 1 year ago

As of now, we configure each pipeline deployment to use 1.5GB of memory.

With this issue, we should evaluate how much memory should be allocated to each deployment as a standard.

flippingbits commented 1 year ago

From my experience, we can reduce the requested memory to 300MB for non-native pipelines, and might be able to further reduce the requests for native pipelines.

WDYT, @ChrisRousey @HknLof