weaveworks / eks-quickstart-app-dev

Example flux manifests for eksctl gitops
Other
56 stars 43 forks source link

Update Amazon CloudWatch manifests to latest version #13

Closed marccarre closed 5 years ago

marccarre commented 5 years ago

Fixes #12. Follows up on #3.

Why?

panic: /rootfs/proc doesn't exists, please use latest yaml templates to launch cloudwatch-agent

How?

eks-gitops-example's manifests were all updated with the latest manifests from:

Only the following ones had changed:

Before

$ kubectl -n amazon-cloudwatch logs --follow cloudwatch-agent-58t6b
[...]
2019-08-23T12:37:20Z I! k8sapiserver OnStartedLeading: ip-192-168-62-114.ap-northeast-1.compute.internal

panic: /rootfs/proc doesn't exists, please use latest yaml templates to launch cloudwatch-agent

goroutine 42 [running]:
github.com/influxdata/telegraf/plugins/processors/k8sdecorator/stores.newNodeInfo(0xc0009c2ba0)
    /local/p4clients/pkgbuild-eAgXi/workspace/src/CWAgent/src/github.com/influxdata/telegraf/plugins/processors/k8sdecorator/stores/nodeinfo.go:105 +0x22e
github.com/influxdata/telegraf/plugins/processors/k8sdecorator/stores.NewPodStore(0xc000342701, 0xe, 0xc00099aa00, 0xffffffffffffffff)
    /local/p4clients/pkgbuild-eAgXi/workspace/src/CWAgent/src/github.com/influxdata/telegraf/plugins/processors/k8sdecorator/stores/podstore.go:66 +0x80
github.com/influxdata/telegraf/plugins/processors/k8sdecorator.(*K8sDecorator).start(0xc00040c000)
    /local/p4clients/pkgbuild-eAgXi/workspace/src/CWAgent/src/github.com/influxdata/telegraf/plugins/processors/k8sdecorator/k8sdecorator.go:68 +0x7b
github.com/influxdata/telegraf/plugins/processors/k8sdecorator.(*K8sDecorator).Apply(0xc00040c000, 0xc00042e250, 0x1, 0x1, 0xc0009c2b40, 0x1, 0x1)
    /local/p4clients/pkgbuild-eAgXi/workspace/src/CWAgent/src/github.com/influxdata/telegraf/plugins/processors/k8sdecorator/k8sdecorator.go:34 +0x325
github.com/influxdata/telegraf/internal/models.(*RunningProcessor).Apply(0xc000156750, 0xc00042e230, 0x1, 0x1, 0xc00042e230, 0x1, 0x1)
    /local/p4clients/pkgbuild-eAgXi/workspace/src/CWAgent/src/github.com/influxdata/telegraf/internal/models/running_processor.go:40 +0x131
github.com/influxdata/telegraf/agent.(*Agent).flusher(0xc00000e070, 0xc0000881e0, 0xc00007ede0, 0x0, 0x0)
    /local/p4clients/pkgbuild-eAgXi/workspace/src/CWAgent/src/github.com/influxdata/telegraf/agent/agent.go:355 +0x146
github.com/influxdata/telegraf/agent.(*Agent).Run.func1(0xc000345730, 0xc00000e070, 0xc0000881e0, 0xc00007ede0)
    /local/p4clients/pkgbuild-eAgXi/workspace/src/CWAgent/src/github.com/influxdata/telegraf/agent/agent.go:403 +0x6d
created by github.com/influxdata/telegraf/agent.(*Agent).Run
    /local/p4clients/pkgbuild-eAgXi/workspace/src/CWAgent/src/github.com/influxdata/telegraf/agent/agent.go:401 +0x5be
$ kubectl get po -n amazon-cloudwatch
NAME                       READY   STATUS             RESTARTS   AGE
cloudwatch-agent-58t6b     0/1     CrashLoopBackOff   10         38m

After

$ kubectl -n amazon-cloudwatch logs --follow cloudwatch-agent-4fcqh
[...]
2019-08-23T13:13:37Z I! k8sapiserver OnStartedLeading: ip-192-168-62-114.ap-northeast-1.compute.internal
2019-08-23T13:13:45Z W! It is the 0 time, going to sleep 200ms before retrying.
$ kubectl get po -n amazon-cloudwatch
NAME                       READY   STATUS    RESTARTS   AGE
cloudwatch-agent-4fcqh     1/1     Running   0          6m5s
Screenshot 2019-08-23 at 22 15 37 Screenshot 2019-08-23 at 22 24 34 Screenshot 2019-08-23 at 22 25 49 Screenshot 2019-08-23 at 22 26 52