google / turbinia

Automation and Scaling of Digital Forensics Tools
Apache License 2.0
746 stars 163 forks source link

Add support for multi-pools for workers #17

Open aarontp opened 7 years ago

aarontp commented 7 years ago

We can add different worker pools that are created to match individual job types (e.g. so Plaso can have worker nodes that have more cpu than workers for other job types). Right now it's one global worker pool.

wajihyassine commented 1 year ago

From a GKE autoscaling perspective, we can consider:

Also handy resource: https://medium.com/google-cloud/scale-your-kubernetes-cluster-to-almost-zero-with-gke-autoscaler-9c78051cbf40

jleaniz commented 1 year ago

Tagging 'celery' because I think we can use built-in functionality for this