go-graphite / go-carbon

Golang implementation of Graphite/Carbon server with classic architecture: Agent -> Cache -> Persister
MIT License
801 stars 126 forks source link

response does not respect aggregation function for request spanning multiple aggregation periods #556

Open ccurtit opened 1 year ago

ccurtit commented 1 year ago

Hi people, I've the following config :

storage-schemas.conf

# this is my first pattern
[statsd]
pattern = ^stats\..*
retentions = 10s:6h,1m:30d,1h:5y
compressed = false

# many other patterns

storage-aggregation.conf

# this is my first pattern
[stats.counters.counts]
pattern = ^stats\.counters\..*\.count$
xFilesFactor = 0.1
aggregationMethod = sum

# many other patterns

Then I run this "backfilling" request :

for ((i=1; i<=6000; i+=1)) ; do echo $i ; echo "stats.counters.backfill.count $(((42+$i)))date -d "now - $i seconds" +%s" | nc -q 0 localhost 2003; done;

so this fills up about 2hours of data, from now til now-2hours

When I request the data from go-carbon : http://localhost:8080/render/?format=json&from=1684155422&target=stats.counters.backfill.count&until=1684164022 I do get the right stuff out


name | "stats.counters.backfill.count"
startTime | 1684155430
stopTime | 1684164030
stepTime | 10
values |  
0 | 0
1 | 0
2 | 0
3 | 0

108 | 46
109 | 6040
110 | 6030
111 | 6026
112 | 6010
113 | 6000
114 | 5991
115 | 6042

now I then back fill on the second "data range" of my storage configuration

for ((i=360; i<=720; i+=10)) ; do echo $i ; echo "stats.counters.backfill.count $(((360+$i)))date -d "now - $i minutes" +%s" | nc -q 0 localhost 2003; done;

Then I ask for a render spanning both ranges

http://localhost:8080/render/?format=json&from=1684135422&target=stats.counters.backfill.count&until=1684164022

and the "head" of my data, which should be summed, because render has decided to shrink the number of points (step is 60seconds) is now not summed but averaged.

name | "stats.counters.backfill.count"
startTime | 1684135440
stopTime | 1684164060
stepTime | 60

347 | 0
348 | 0
349 | 0
350 | 0
351 | 4830.4
352 | 6016.833333333333
353 | 5979.5
354 | 5917.5
355 | 5858.666666666667
356 | 5797

Am I misconfiguring something, or is this a bug ? Given my config, I was expecting something along the lines of

name | "stats.counters.backfill.count"
startTime | 1684135440
stopTime | 1684164060
stepTime | 60

347 | 0
348 | 0
349 | 0
350 | 0
351 | 36000
352 | 35400
353 | 34800
354 | subtract 600
355 | etc..
356 | etc...

I'm running go-carbon 0.17.1