Use `region` field as comma-delimited node selector for k8s

[sjawhar]

• Adds the ability to use the region field of the task in the Terraform file to pass node selector labels to the job.
• Expected format is comma-delimited key=value pairs
  • e.g. region: foo=bar,goo=baz,...
  • Hacky?

[0x2b3bfa0]

Hacky?

Yes, cough, cough, yes: https://github.com/iterative/terraform-provider-iterative/issues/412#issuecomment-1159729184. Nevertheless, sounds useful to me.

[dacbd]

willing to add some documentation snippets?

[sjawhar]

Yes, cough, cough, yes: #412 (comment). Nevertheless, sounds useful to me.

I got the impression from the convo on my last PR that we didn't want to add cloud-specific options, but I'm happy to decline this PR and do it that way if preferred :smile:

[0x2b3bfa0]

We have some mixed opinions on the topic, but this proposal sounds good to me. It's a stretch of the “region” concept, but at least it adds some support for k8s node selectors instead of outright ignoring the region field.

[sjawhar]

It looks like the smoke test failed because of "Quota Exceeded"

[omesser]

It's a stretch of the “region” concept, but at least it adds some support for k8s node selectors instead of outright ignoring the region field.

Yes indeed :) I have no strong objection to merging this (even if it becomes a small tech debt..), but afterwards - let's please discuss proper support for k8s specific configuration (it's not a cloud provider), cc @iterative/cml I am not convinced we're doing anything other than delaying the inevitable and confusing users with mixing up of terminologies 🥲

[0x2b3bfa0]

Also passed the Kubernetes smoke test :shipit: