cirruslabs / orchard

Orchestrator for running Tart Virtual Machines on a cluster of Apple Silicon devices
Other
192 stars 15 forks source link

Failed to reactively sync VMs on worker #118

Closed ruimarinho closed 11 months ago

ruimarinho commented 1 year ago

Hi,

Under unknown circumstances, a worker can stop syncing with the controller with the following error:

{"level":"error","ts":1694015012.3058627,"caller":"scheduler/scheduler.go:144","msg":"Failed to reactively sync VMs on worker foobar: no worker registered with this name: foobar","stacktrace":"github.com/cirruslabs/orchard/internal/controller/scheduler.(*Scheduler).schedulingLoopIteration\n\t/tmp/orchard/internal/controller/scheduler/scheduler.go:144\ngithub.com/cirruslabs/orchard/internal/controller/scheduler.(*Scheduler).Run\n\t/tmp/orchard/internal/controller/scheduler/scheduler.go:68"}

Any hint into what can cause this issue?

edigaryev commented 11 months ago

I'm pretty sure this is the same issue as https://github.com/cirruslabs/orchard/issues/126 because it's essence is that the worker is not being registered on the gRPC-side when it should be.

So I will close this one to track everything in https://github.com/cirruslabs/orchard/issues/126.