Closed perederyaev closed 1 year ago
Describe the bug When I have maxBatchSize=0 and upstream protocol carbonapi_v3_pb ConsolidateBy function always uses average for aggregation.
CarbonAPI Version This issue is reproducible on current master.
CarbonAPI Configuration:
listen: "0.0.0.0:8081" upstreams: backendsv2: backends: - groupName: "test" protocol: "carbonapi_v3_pb" lbMethod: "broadcast" maxBatchSize: 0 servers: ["http://x.x.x.x:9090"] expireDelaySec: 0 logger: - logger: "" file: "/var/log/carbonapi.log" level: "debug" encoding: "console" encodingTime: "iso8601" encodingDuration: "seconds"
Simplified query (if applicable) curl "localhost:8081/render?format=json&target=consolidateBy(carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded,'sum')&from=18%3A08_20230725&until=18%3A17_20230725&maxDataPoints=5"
Backend response (if possible)
Consolidation is not applied as maxDataPoints is greater than metrics number in the range
$ curl "localhost:8081/render?format=json&target=consolidateBy(carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded,'sum')&from=18%3A08_20230725&until=18%3A17_20230725&maxDataPoints=50" | jq [ { "target": "consolidateBy(carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded,\"sum\")", "datapoints": [ [ 58, 1690308480 ], [ 59, 1690308540 ], [ 60, 1690308600 ], [ 59, 1690308660 ], [ 60, 1690308720 ], [ 59, 1690308780 ], [ 60, 1690308840 ], [ 60, 1690308900 ], [ 59, 1690308960 ], [ 59, 1690309020 ] ], "tags": { "consolidateBy": "sum", "name": "carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded" } } ]
maxBatchSize=0 - avg is used ignoring provided sum function
$ curl "localhost:8081/render?format=json&target=consolidateBy(carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded,'sum')&from=18%3A08_20230725&until=18%3A17_20230725&maxDataPoints=5" | jq [ { "target": "consolidateBy(carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded,\"sum\")", "datapoints": [ [ 58.5, 1690308480 ], [ 59.5, 1690308600 ], [ 59.5, 1690308720 ], [ 60, 1690308840 ], [ 59, 1690308960 ] ], "tags": { "consolidateBy": "sum", "name": "carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded" } } ]
maxBatchSize=1 - sum is used like it should be
$ curl "localhost:8081/render?format=json&target=consolidateBy(carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded,'sum')&from=18%3A08_20230725&until=18%3A17_20230725&maxDataPoints=5" | jq [ { "target": "consolidateBy(carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded,\"sum\")", "datapoints": [ [ 117, 1690308480 ], [ 119, 1690308600 ], [ 119, 1690308720 ], [ 120, 1690308840 ], [ 118, 1690308960 ] ], "tags": { "consolidateBy": "sum", "name": "carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded" } } ]
Additional context If use protocol: "carbonapi_v2_pb" it works well in spite of maxBatchSize setting
Looks like issue is in graphite-clickhouse not in carbonapi. In our case graphite-clickhouse has internal-aggregation enabled and uses "avg" as rollup-default-function .
Describe the bug When I have maxBatchSize=0 and upstream protocol carbonapi_v3_pb ConsolidateBy function always uses average for aggregation.
CarbonAPI Version This issue is reproducible on current master.
CarbonAPI Configuration:
Simplified query (if applicable) curl "localhost:8081/render?format=json&target=consolidateBy(carbon.agents.04f2e201fc3b.upload.graphite_tagged.uploaded,'sum')&from=18%3A08_20230725&until=18%3A17_20230725&maxDataPoints=5"
Backend response (if possible)
Consolidation is not applied as maxDataPoints is greater than metrics number in the range
maxBatchSize=0 - avg is used ignoring provided sum function
maxBatchSize=1 - sum is used like it should be
Additional context If use protocol: "carbonapi_v2_pb" it works well in spite of maxBatchSize setting