search

terascope / teraslice

Scalable data processing pipelines in JavaScript
https://terascope.github.io/teraslice/
Apache License 2.0
50 stars 13 forks source link

Proper readyForAllocation in k8s #744

Open godber opened 6 years ago

godber commented 6 years ago

For the initial first pass of implementing support for kubernetes I am going to punt on implementing readyForAllocation. This issue should serve as a reminder that the k8s clustering doesn't currently provide any feedback to users if they submit a job to an oversubscribed cluster.

According to the k8s resource management docs the current behavior will be:

If the scheduler cannot find any node where a Pod can fit, the Pod remains unscheduled until a place can be found.

It might not work this way until the pods explicitly request resources.

(Refs PR #738)

kstaken commented 6 years ago

I guess this isn't surprising. Teraslice native actually works the same way. It means the job queue in Teraslice will never have any effect as it will always be handed off to Kubernetes.

godber commented 4 years ago

At the very least the Teraslice master could ping the k8s API somehow.