search

komljen / helm-charts

Helm charts - Apps for Kubernetes
154 stars 102 forks source link

fluent-bit failing to get kubernetes medata #2

Closed deas closed 6 years ago

deas commented 6 years ago

I am trying to follow your post Get Kubernetes Logs with EFK Stack in 5 Minutes using minikube 0.28.1/kubernetes 1.10.0 and I was/am running in a few issues:

First thing was that

helm install --name es-operator \
    --namespace logging \
    --set rbac.create=false \
    akomljen-charts/elasticsearch-operator

failed to set up the CRD. Container log was showing:

time="2018-08-03T05:03:07Z" level=info msg="elasticsearch operator starting up!"
time="2018-08-03T05:03:07Z" level=info msg="Using Variables:"
time="2018-08-03T05:03:07Z" level=info msg="   baseImage: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0"
time="2018-08-03T05:03:07Z" level=info msg="Using InCluster k8s config"
panic: customresourcedefinitions.apiextensions.k8s.io "elasticsearchclusters.enterprises.upmc.com" is forbidden: User "system:serviceaccount:logging:es-operator-elasticsearch-operator" cannot get customresourcedefinitions.apiextensions.k8s.io at the cluster scope

goroutine 1 [running]:
github.com/upmc-enterprises/elasticsearch-operator/pkg/k8sutil.(*K8sutil).CreateKubernetesCustomResourceDefinition(0xc420191620, 0x1045e88, 0x6)
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/k8sutil/k8sutil.go:218 +0x64a
github.com/upmc-enterprises/elasticsearch-operator/pkg/controller.(*Controller).init(0xc420171e68, 0x1046fa7, 0xc420171db0)
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/controller/controller.go:69 +0x32
github.com/upmc-enterprises/elasticsearch-operator/pkg/controller.(*Controller).Run(0xc420171e68, 0x1045e88, 0x6)
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/controller/controller.go:58 +0x2f
main.Main(0xc4200ac058)
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/cmd/operator/main.go:111 +0x58e
main.main()
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/cmd/operator/main.go:144 +0x22

which looks like an auth issue to me.

Downloaded 0.2 at https://raw.githubusercontent.com/upmc-enterprises/elasticsearch-operator/master/example/controller.yaml, changed the namespace to logging and deployed it. That created the CRD and

helm install --name efk \
    --namespace logging \
    akomljen-charts/efk

installed the operator.

However, fluent-bit fails to get POD metadata from the apiserver:

[2018/08/03 05:16:11] [ info] [engine] started (pid=1)
[2018/08/03 05:16:11] [ info] [filter_kube] https=1 host=kubernetes.default.svc port=443
[2018/08/03 05:16:11] [ info] [filter_kube] local POD info OK
[2018/08/03 05:16:11] [ info] [filter_kube] testing connectivity with API server...
[2018/08/03 05:16:11] [ warn] [filter_kube] could not get meta for POD fluent-bit-2vvwm

Turned out to be an auth issue and --set fluent-bit.rbac.create=true did the trick.