cirruslabs / orchard

Orchestrator for running Tart Virtual Machines on a cluster of Apple Silicon devices
Other
189 stars 13 forks source link

Support labelSelect for vm scheduler #154

Open gsakun opened 5 months ago

gsakun commented 5 months ago

Currently,virtual machines can only be scheduled to appropriate worker based on resource. I hope schedule virtual machines to relevant worker based on labelselect like k8s pod.I would like to do this task.

fkorotkov commented 5 months ago

What is your use case? We though about adding node selection down the line but haven't figured a use case. In most cases people have identical nodes in the cluster (bunch of Mac Minis).

gsakun commented 5 months ago

What is your use case? We though about adding node selection down the line but haven't figured a use case. In most cases people have identical nodes in the cluster (bunch of Mac Minis).

In our usage scenario, we want CI tasks to be fixed on certain nodes as much as possible in order to reuse the cache efficiently.

Some projects have a high priority and require dedicated machines. The remaining projects can reuse the machines.

edigaryev commented 5 months ago

In our usage scenario, we want CI tasks to be fixed on certain nodes as much as possible in order to reuse the cache efficiently.

You can easily achieve this with resource management.

Simply provide a single resource to a subset of workers (e.g. tart-vms-high-priority: 2) and then create VMs that specifically request this resource (orchard create vm --resources tart-vms-high-priority=1).

This will ensure that these VMs will only be scheduled on a subset of desired workers.

fkorotkov commented 5 months ago

You can even introduce an artificial "project-name-cache" resource on some nodes to gurantee that cache of a certain project will be available there.

gsakun commented 5 months ago

In our usage scenario, we want CI tasks to be fixed on certain nodes as much as possible in order to reuse the cache efficiently.

You can easily achieve this with resource management.

Simply provide a single resource to a subset of workers (e.g. tart-vms-high-priority: 2) and then create VMs that specifically request this resource (orchard create vm --resources tart-vms-high-priority=1).

This will ensure that these VMs will only be scheduled on a subset of desired workers.

Resources are already set when the worker is initialized and cannot be dynamically configured. And not flexible enough.

gsakun commented 5 months ago

You can even introduce an artificial "project-name-cache" resource on some nodes to gurantee that cache of a certain project will be available there.

Therefore, it is necessary to schedule the vm at the specified node in combination with the selection criteria

edigaryev commented 5 months ago

Therefore, it is necessary to schedule the vm at the specified node in combination with the selection criteria

Would you mind providing a minimalistic example to demonstrate your point? I.e. which labels you'd set on workers and which labels you'd use when creating VMs.

gsakun commented 5 months ago

Therefore, it is necessary to schedule the vm at the specified node in combination with the selection criteria

Would you mind providing a minimalistic example to demonstrate your point? I.e. which labels you'd set on workers and which labels you'd use when creating VMs.

Yes, what I want to implement is to schedule the vm to a certain machine based on the labelSelect. for example,Now I own four MacMini.I labeled the first two machines role=test,When I create the vm, I set the selection condition of role=test to ensure that the vm will only be scheduled to the machine that has this label