deis-monitor-telegraf shows - error getting disk usage info: too many levels of symbolic links

deis / monitor

Monitoring for Deis Workflow

https://deis.com

MIT License

22 stars 32 forks source link

deis-monitor-telegraf shows - error getting disk usage info: too many levels of symbolic links #163

Open rimusz opened 7 years ago

rimusz commented 7 years ago

deis-monitor-telegraf-rz92o deis-monitor-telegraf 2016/12/02 16:03:30 E! ERROR in input [inputs.disk]: error getting disk usage info: too many levels of symbolic links

bacongobbler commented 7 years ago

Can you please provide some basic information on how you are seeing this:

What version of workflow?
What version of Kubernetes?
How is it deployed?
Is there a set of instructions to re-create this issue?
Have you gone through the troubleshooting guide?

rimusz commented 7 years ago

latest Workflow v2.8.0 deployed with Helm chart
k8s v1.4.6
deployed on AWS, CoreOS, similar to tack setup @gerred can explain more on the setup
no, there are no set of instructions to re-create this issue
yes, only error I got is deis-monitor-telegraf-rz92o deis-monitor-telegraf 2016/12/02 16:03:30 E! ERROR in input [inputs.disk]: error getting disk usage info: too many levels of symbolic links

wetoolaguer commented 7 years ago

Hi, I experienced the same issue.

Workflow versions v2.9.0, v2.5.0 with helm and helmc
deployed on AWS, debian / ubuntu via kops
I get 2016/12/14 11:43:10 ERROR in input [disk]: error getting disk usage info: too many levels of symbolic links and one of my telegrafs intermittently fails with this log
```
Creating topic with URL: http://100.71.131.206:4151/topic/create?topic=metrics
```

bacongobbler commented 7 years ago

relevant: https://github.com/influxdata/telegraf/issues/1352

bacongobbler commented 7 years ago

This does appear to be an issue upstream with systemd, perhaps with https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1555760.

mboersma commented 7 years ago

There are some workarounds suggested in influxdata/telegraf#1352 that users can try. A pervasive fix requires Ubuntu to make changes to systemd or binfmt, which is something many people are still waiting for sadly.

I'm going to move this to the next milestone since there's nothing currently actionable for Workflow here IMHO.

monaka commented 7 years ago

@vdice Was this issue fixed on v2.12 ? Or re-tagged on v2.13 or later?

bacongobbler commented 7 years ago

This issue was never resolved; see above

monaka commented 7 years ago

Ah, I see.

michalkm commented 7 years ago

Any updates or workarounds on this matter ?

jchauncey commented 7 years ago

See earlier in the thread. As it stands this is not a fix that we can make on the workflow end it has to be fixed in an upstream project.