search

prometheus-junkyard / mesos_exporter

Prometheus exporter for Mesos, deprecated.
Apache License 2.0
42 stars 22 forks source link

Task data continues to be reported after a task is gone #2

Closed barkerd427 closed 9 years ago

barkerd427 commented 9 years ago

I'm not sure why, but when I look at the Graph in Prometheus of mesos_task_cpus_system_time_secs or mesos_task_mem_rss_bytes I get static results from tasks that are long since gone. I think all mesos prefixed metrics do the same thing. What I mean by static is a solid horizontal line running across the entire graph as if the exporter is providing the last value for that task. When I look at the actual /metrics endpoint, I do see that all tasks are reported when I only have a couple running.

antonlindstrom commented 9 years ago

I've noticed this as well, I'm not really sure how to solve this. I'll try to take a look and see if I can somehow timeout tasks that no longer exists.

Thanks!

barkerd427 commented 9 years ago

I'll try to look at the code also.

On Sun, Mar 1, 2015 at 2:36 AM, Anton Lindström notifications@github.com wrote:

I've noticed this as well, I'm not really sure how to solve this. I'll try to take a look and see if I can somehow timeout tasks that no longer exists.

Thanks!

— Reply to this email directly or view it on GitHub https://github.com/antonlindstrom/mesos_exporter/issues/2#issuecomment-76587998 .

barkerd427 commented 9 years ago

It looks like the endpoints return an empty array when there is nothing running on the slave, so we just need to remove the gauge perhaps when the task is no longer running.