search

GoogleCloudPlatform / flink-on-k8s-operator

[DEPRECATED] Kubernetes operator for managing the lifecycle of Apache Flink and Beam applications.
Apache License 2.0
658 stars 265 forks source link

supported NodeSelector for job #201

Open Mrart opened 4 years ago

Mrart commented 4 years ago

NodeSelector is for pod schedule，but now is only support NodeSelector for tm/jm pod

functicons commented 4 years ago

Could you make the change? Should be straightforward.

Mrart commented 4 years ago

job not support 'selector'? I will push the two function togather.I will do it. I mean job selector not use as like jobmanger deploy selector.

Mrart commented 4 years ago

@functicons ok,anthor thing. If we want use labels for schedule as our k8s can scheduler with it. but the deployment/job/services etcs. not add the flinkcluster metadata.labels. So It can not match the schedulers. What do you thing about it, We can set flinkcluster metadata.labels to all workloads?

functicons commented 4 years ago

@Mrart sorry, I don't fully understand your request. IIUC, you want to be able to apply nodeSelector to Job (submitter) of a Flink cluster, right? I think we can add a property in the jobSpec of the CR and translate it to the podSpec of the job resource. But nodeSelector only applies to Pods, it doesn't apply to Services.

Let me know if I misunderstood anything.

Mrart commented 4 years ago

@functicons I want add global nodeseletor for cr， don't need use it for tm selector/ jm selector /job selector. Just one entry to set nodeselector for all workload， Do you agree this?

Mrart commented 4 years ago

@Mrart sorry, I don't fully understand your request. IIUC, you want to be able to apply nodeSelector to Job (submitter) of a Flink cluster, right? I think we can add a property in the jobSpec of the CR and translate it to the podSpec of the job resource. But nodeSelector only applies to Pods, it doesn't apply to Services.

Let me know if I misunderstood anything.

Ingore this, jobSpec support selector has some problem.

functicons commented 4 years ago

What's the problem?

Mrart commented 4 years ago

Job spec user selector ,need add manulSelector:true.

guruprasathT commented 4 years ago

Hello, I tried locally with changes jobSpec, manualSelector:true but stopped with this error (selector does not match template labels) on creating this job pod. Any clue on this will be helpful. Thanks !

2020-09-09T21:43:21.983Z        INFO    controllers.FlinkCluster        Failed to created job   {"cluster": "topic-splitter/sampler-beta", "error": "Job.batch \"sampler-beta-job\" is invalid: [spec.selector: Required value, spec.template.metadata.labels: Invalid value: map[string]string{\"app\":\"flink\", \"cluster\":\"sampler-beta\"}: `selector` does not match template `labels`]"}