[gmetad] `num` data source wrong type?

dennisklein commented 8 years ago

I believe it is wrong to set the num data source equal to the type of the sum data source ( gmetad/rrd_helpers.c#L280). Should it not rather always be of type "GAUGE"?

vvuksan commented 8 years ago

That's a good point. Is this causing any issues ?

dennisklein commented 8 years ago

That's a good point. Is this causing any issues ?

Yes, on counter-type DSTs the stored num value does not reflect the number of hosts any more, but the change rate, e.g.

lxmon13:/var/lib/ganglia/rrds/kronos/__SummaryInfo__# RRDCACHED_ADDRESS=unix:/var/run/rrdcached/rrdcached.sock rrdtool fetch -s end-60s lustre_client_nyx_write_bytes.rrd AVERAGE
                            sum                 num

1452274965: 6.7345256112e+17 0.0000000000e+00
1452274980: 6.7345256112e+17 0.0000000000e+00
1452274995: -nan -nan
1452275010: -nan -nan
1452275025: -nan -nan

p.s.: The sum data points in my summary RRDs are also way too high, but I have not figured out yet why. I suspect some overflow... This issue I just discovered while looking for the cause of the wrong sums.

vvuksan commented 8 years ago

Makes sense. Did you compile the new version with your change and can confirm it fixes the issue ?

dennisklein commented 8 years ago

I will confirm my fix latest on Monday and report back here.

dennisklein commented 8 years ago

Successfully compiled ab3d7de and manually verified by sending a test metric with slope positive:

$ rrdtool fetch -s end-60s rrds/__SummaryInfo__/XXX.rrd AVERAGE
                            sum                 num

1452519120: 1.4074072667e+06 1.2666666667e+00
1452519135: 2.2222220000e+06 2.0000000000e+00
1452519150: 2.2222220000e+06 2.0000000000e+00
1452519165: -nan -nan
1452519180: -nan -nan
$ rrdtool info rrds/__SummaryInfo__/XXX.rrd | grep "^ds\[num\]\.type"
ds[num].type = "GAUGE"

ganglia / monitor-core

[gmetad] `num` data source wrong type? #240