search

volcano-sh / volcano

A Cloud Native Batch System (Project under CNCF)
https://volcano.sh
Apache License 2.0
4.03k stars 930 forks source link

NodeSelector not filtering nodes correctly in volcano-scheduler v1.8.2+ #3516

Closed 1wolfalone1 closed 3 weeks ago

1wolfalone1 commented 2 months ago

What happened: I attempted to configure multiple schedulers in Volcano and encountered unexpected behavior with different versions of the volcano-scheduler image.

When I set the volcano-scheduler image to version 1.8.2 (or the default latest in the Helm deployment), image I configured a nodeSelector with volcano.sh/nodegroup-name=g2, which should match two nodes: agent2 and agent3. image image However, the Volcano scheduler logs indicated that it was collecting all nodes in the cluster instead of just those matching the configured node selector. image

This led me to believe there might be an error in my configuration. However, downgrading the volcano-scheduler image to version 1.8.1 or below (within the Helm deployment) resolved the issue. image The logs confirmed this behavior, showing that only nodes agent2 and agent3 were collected, and resource calculation for the scheduler in proportion plugins functioned correctly. image What you expected to happen: The ability to create multiple schedulers with a nodeSelector should function correctly from version 1.8.2 of the volcano-scheduler image onwards.

Environment:

Monokaix commented 2 months ago

Did you also use nodegroup plugin or just multi schedulers?

1wolfalone1 commented 2 months ago

As a test, I'm currently using multi-schedulers with a node selector in that example. If this approach proves successful, I'll then activate the node group later.

Monokaix commented 2 months ago

There is truly a probelm in versiob v1.8.2，same issue https://github.com/volcano-sh/volcano/issues/3444 , will fix it later: )

Monokaix commented 2 months ago

/good-first-issue

volcano-sh-bot commented 2 months ago

@Monokaix: This request has been marked as suitable for new contributors.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-good-first-issue command.

In response to [this](https://github.com/volcano-sh/volcano/issues/3516): >/good-first-issue Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
Prepmachine4 commented 4 weeks ago

I am interested in this issue, can you assign it to me?

Monokaix commented 4 weeks ago

I am interested in this issue, can you assign it to me?

Thanks for your warming! It has been fixed by https://github.com/volcano-sh/volcano/pull/3594，you can still select other good first issue to contribute: )

Monokaix commented 3 weeks ago

/close

volcano-sh-bot commented 3 weeks ago

@Monokaix: Closing this issue.

In response to [this](https://github.com/volcano-sh/volcano/issues/3516#issuecomment-2254308527): >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.