search

admiraltyio / admiralty

A system of Kubernetes controllers that intelligently schedules workloads across clusters.
https://admiralty.io
Apache License 2.0
683 stars 86 forks source link

node keep deleting #155

Closed lxm closed 1 year ago

lxm commented 1 year ago

vk node will be delete with follow log 

E1227 09:58:05.545043       1 controller.go:117] error syncing 'admiralty-default-lobby-gpu-test001-2bc45747aa': node "admiralty-default-lobby-gpu-test001-2bc45747aa" not found, requeuing
E1227 09:58:06.185856       1 controller.go:117] error syncing 'admiralty-default-lobby-gpu-test001-2bc45747aa': node "admiralty-default-lobby-gpu-test001-2bc45747aa" not found, requeuing
E1227 09:58:07.466519       1 controller.go:117] error syncing 'admiralty-default-lobby-gpu-test001-2bc45747aa': node "admiralty-default-lobby-gpu-test001-2bc45747aa" not found, requeuing
E1227 09:58:10.026652       1 controller.go:117] error syncing 'admiralty-default-lobby-gpu-test001-2bc45747aa': node "admiralty-default-lobby-gpu-test001-2bc45747aa" not found, requeuing
time="2022-12-27T09:58:13Z" level=error msg="failed to update node lease" error="Operation cannot be fulfilled on leases.coordination.k8s.io \"admiralty-default-lobby-gpu-test001-2bc45747aa\": StorageError: invalid object, Code: 4, Key: /registry/leases/kube-node-lease/admiralty-default-lobby-gpu-test001-2bc45747aa, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: c8abac0a-3059-42f1-bce6-3e081d9f86e9, UID in object meta: "
E1227 09:58:15.147385       1 controller.go:117] error syncing 'admiralty-default-lobby-gpu-test001-2bc45747aa': node "admiralty-default-lobby-gpu-test001-2bc45747aa" not found, requeuing
adrienjt commented 1 year ago

I would need more details about your environment to help with this.