search

gruntwork-io / kubergrunt

Kubergrunt is a standalone go binary with a collection of commands to fill in the gaps between Terraform, Helm, and Kubectl. https://www.gruntwork.io
Apache License 2.0
516 stars 89 forks source link

CoreDNS 1.8.3 update requires clusterrole patch #129

Closed gaba-mlindner closed 3 years ago

gaba-mlindner commented 3 years ago

I just upgraded my lab EKS cluster (managed by Gruntwork's terraform-aws-eks modules) from 1.19 to 1.20 and ran into some issues with kubergrunt's CoreDNS update.

After the module's provisioner had executed eks sync-core-components .., DNS resolution inside the cluster stopped working and the new CoreDNS pods starting logging errors like:

E0611 10:20:52.181166       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.20.2/tools/cache/reflector.go:167: Failed to watch *v1beta1.EndpointSlice: failed to list *v1beta1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:serviceaccount:kube-system:coredns" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope

Quick googling led me to AWS' docs (see step 5): https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html#updating-coredns-add-on

It turns out that the CoreDNS 1.8.3 update requires additional permissions.

Manually patching the system:coredns clusterrole resolved my issues, but I think it would make sense to have kubergrunt handle this kind of thing automatically (since it's already patching the image version and configmap anyways).

versions: terraform v1.0.0 terragrunt v0.29.2 kubergrunt v0.7.1 terraform-aws-eks v0.41.0

yorinasub17 commented 3 years ago

Thanks for the report! I have opened a PR with the fix: https://github.com/gruntwork-io/kubergrunt/pull/130

rhoboat commented 3 years ago

Looking at the related PR and tests now!

yorinasub17 commented 3 years ago

This is now handled in https://github.com/gruntwork-io/kubergrunt/releases/tag/v0.7.2

gaba-mlindner commented 3 years ago

Just FYI, I upgraded another cluster with the v0.7.2 version last night and it worked like a charm without manual intervention. Thanks for the quick fix!

yorinasub17 commented 3 years ago

Thanks for closing the loop!