knative / serving

Kubernetes-based, scale-to-zero, request-driven compute
https://knative.dev/docs/serving/
Apache License 2.0
5.57k stars 1.16k forks source link

Using nodeSelector/tolerations with Serving? #1816

Closed jeremyrickard closed 4 years ago

jeremyrickard commented 6 years ago

👋 This is kind of part question/feature request!

Could Revision be extended to support using Tolerations and/or Node Selectors? I get the reason why it doesn't include the entire pod spec, but I think some additional attributes to influence scheduling could be useful. I looked through the current types and I don't see how I can do this today, but I might have missed something!

For example, I might care about where my thing gets scheduled if....

In most cases, I probably don't care where things run because serverless...but somettimes I think there are. This change seems like it would be useful and not have a ton of impact to user experience or complexity.

I'd be interested in working on this if it's something that seems palatable. cc: @krancour

duglin commented 4 years ago

@mattmoor let me check....

duglin commented 4 years ago

@mattmoor @mpetason will take a look at it

mattmoor commented 4 years ago

Awesome, thanks @duglin! :party-matt:

dprotaso commented 4 years ago

/unassign @jeremyrickard /assign @mpetason

calclavia commented 4 years ago

Any updates on this issue? For example, it would be great to use KNative to schedule on only preemptive nodes on GKE.

emaildanwilson commented 4 years ago

Would config-features then have a flag to allow affinity/nodeSelector/etc in the podSpec or would those settings somehow get set from somewhere else as defined with a fieldRef? I'm trying to understand what the implementation would look like.

JRBANCEL commented 4 years ago

Would config-features then have a flag to allow affinity/nodeSelector/etc in the podSpec

Yes, this looks like the right way to do this.

emaildanwilson commented 4 years ago

/assign @emaildanwilson

mpetason commented 4 years ago

Talked to @emaildanwilson on Slack. They want to take over the issue and work on it. I'm on vacation next week so I won't be able to finish it as quickly.

evankanderson commented 4 years ago

/assign @whaught

It sounds like Weston might be the right person to do an initial pass and then find someone else with approver if he doesn't have it by then.

evankanderson commented 4 years ago

/unassign whaught

Whoops, this is the issue, not the PR.

rccrdpccl commented 4 years ago

Glad to see this landing to knative-serving. However, since this did not look like it was going to be implemented, we have been implementing it through annotations, one CRD and a webhook, greatly simplifying the definition from the user perspective https://github.com/softonic/node-policy-webhook @sumowrestler @calclavia

sniperking1234 commented 4 years ago

When will this feature be available?

mattmoor commented 4 years ago

in 0.17, which cuts a week from tomorrow.