search

kubernetes-sigs / external-dns

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services
Apache License 2.0
7.76k stars 2.59k forks source link

externalDNS pod runs into crashloopback on k8s 1.20.5 #2438

Closed kalyanbhonagiri16 closed 2 years ago

kalyanbhonagiri16 commented 3 years ago

externalDNS pod runs into crashloopback on k8s 1.20.5:

Pod to be in Running state:

It was working until yesterday and suddenly started seeing go issues on the pod?:

We are using infoblox as our service provider kubectl logs:

kubectl logs -f -n external-dns external-dns-66f574d766-55j98 panic: interface conversion: interface {} is []interface {}, not string

goroutine 1 [running]: github.com/infobloxopen/infoblox-go-client.(EA).UnmarshalJSON(0xc000f2bbf8, 0xc000fba44d, 0xd6, 0xb9b3, 0x7f66515f64d8, 0xc000f2bbf8) /go/pkg/mod/github.com/infobloxopen/infoblox-go-client@v0.0.0-20180606155407-61dc5f9b0a65/objects.go:510 +0x4b7 encoding/json.(decodeState).object(0xc001560dc0, 0x240a7e0, 0xc000f2bbf8, 0x195, 0xc001560de8, 0x7b) /usr/local/go/src/encoding/json/decode.go:609 +0x207c encoding/json.(decodeState).value(0xc001560dc0, 0x240a7e0, 0xc000f2bbf8, 0x195, 0x240a7e0, 0xc000f2bbf8) /usr/local/go/src/encoding/json/decode.go:370 +0x6d encoding/json.(decodeState).object(0xc001560dc0, 0x2585180, 0xc000f2bb98, 0x199, 0xc001560de8, 0x7b) /usr/local/go/src/encoding/json/decode.go:765 +0x12fe encoding/json.(decodeState).value(0xc001560dc0, 0x2585180, 0xc000f2bb98, 0x199, 0x2585180, 0xc000f2bb98) /usr/local/go/src/encoding/json/decode.go:370 +0x6d encoding/json.(decodeState).array(0xc001560dc0, 0x21d5fc0, 0xc00155c1e0, 0x16, 0xc001560de8, 0x5b) /usr/local/go/src/encoding/json/decode.go:558 +0x1aa encoding/json.(decodeState).value(0xc001560dc0, 0x21d5fc0, 0xc00155c1e0, 0x16, 0xc0001fa140, 0x7b94c5) /usr/local/go/src/encoding/json/decode.go:360 +0x105 encoding/json.(decodeState).unmarshal(0xc001560dc0, 0x21d5fc0, 0xc00155c1e0, 0xc001560de8, 0x0) /usr/local/go/src/encoding/json/decode.go:180 +0x1ea encoding/json.Unmarshal(0xc000fa6000, 0x160ad, 0x1fe00, 0x21d5fc0, 0xc00155c1e0, 0x0, 0x0) /usr/local/go/src/encoding/json/decode.go:107 +0x112 github.com/infobloxopen/infoblox-go-client.(Connector).GetObject(0xc0001e6fc0, 0x2af9840, 0xc000299110, 0x0, 0x0, 0x21d5fc0, 0xc00155c1e0, 0xfea310cd32080ee4, 0xe7c11be0430f7d65) /go/pkg/mod/github.com/infobloxopen/infoblox-go-client@v0.0.0-20180606155407-61dc5f9b0a65/connector.go:301 +0x265 sigs.k8s.io/external-dns/provider/infoblox.(InfobloxProvider).zones(0xc0001316c0, 0x40a93f, 0x2327840, 0x1a2fa0, 0x2277fa0, 0x2801) /sigs.k8s.io/external-dns/provider/infoblox/infoblox.go:258 +0x1d7 sigs.k8s.io/external-dns/provider/infoblox.(InfobloxProvider).Records(0xc0001316c0, 0x2b13680, 0xc0003ff880, 0x1, 0x21d5f80, 0xc000af3220, 0x21d5f80, 0xc000af3220) /sigs.k8s.io/external-dns/provider/infoblox/infoblox.go:142 +0x45 sigs.k8s.io/external-dns/registry.(TXTRegistry).Records(0xc000eea600, 0x2b13680, 0xc0003ff880, 0xc0001fb3b8, 0x2, 0x2, 0xc0001fb384, 0x2) /sigs.k8s.io/external-dns/registry/txt.go:82 +0x7b sigs.k8s.io/external-dns/controller.(Controller).RunOnce(0xc00026e6e0, 0x2b13680, 0xc0003ff880, 0x3c74b20, 0x25e7801) /sigs.k8s.io/external-dns/controller/controller.go:126 +0x66 sigs.k8s.io/external-dns/controller.(Controller).Run(0xc00026e6e0, 0x2b13680, 0xc0003ff880) /sigs.k8s.io/external-dns/controller/controller.go:194 +0x20f main.main() /sigs.k8s.io/external-dns/main.go:354 +0xe50 Environment:

sagor999 commented 3 years ago

What infoblox API do you specify? Has anything changed with infoblox setup on your end? Looking at callstack it appears that it is crashing when processing data from infoblox server.

kalyanbhonagiri16 commented 2 years ago

Got confirmation from our infoblox team that nothing was changed on their end. The issue was resolved after upgrading the image to latest version v0.10.1