search

aws / eks-anywhere

Run Amazon EKS on your own infrastructure 🚀
https://anywhere.eks.amazonaws.com
Apache License 2.0
1.96k stars 283 forks source link

Need doc update on best practice to sync mgmt and worker nodes' timestamps using an NTP server #3302

Open madhavvishnubhatta opened 2 years ago

madhavvishnubhatta commented 2 years ago

What happened: When the mgmt and worker nodes' timestamps are out of sync, there are errors like below:

level=error msg=k8sError error="github.com/cilium/cilium/pkg/k8s/watchers/endpoint_slice.go:91: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: Unauthorized" subsys=k8s

What you expected to happen: There must be a note in the documentation that not syncing the timestamps will lead to errors and suggest a best practice to keep the timestamps in sync.

How to reproduce it (as minimally and precisely as possible): I really do not know. I heard this from someone who experienced it and so creating an issue here to keep track of the update that is needed to the documentation.

Anything else we need to know?: There is already a ticket to expose the time-server settings in the cluster spec - #707 which could be a way to do it. At the moment, I think the workaround is to manually do this on the VMs.

Environment:

jonahjon commented 2 years ago

Has this been an issue you've seen on a certain version of EKS-A @madhavvishnubhatta ?

madhavvishnubhatta commented 2 years ago

This issue came up in 0.10.1 but I think the issue can happen in other versions too.

chrisnegus commented 2 years ago

Updates to troubleshooting docs in PR #3303 partially address this issue by identifying the error message that occurs when control plane and worker nodes timestamps are not in sync. However, this issue can stay open until Issue #707 is resolved.