avito-tech / bioyino

High performance and high-precision multithreaded StatsD server
The Unlicense
228 stars 22 forks source link

Incorrect display of "Counting" when sending 0 #82

Open syatihoko opened 9 months ago

syatihoko commented 9 months ago

Hello. When creating a certain metric through "Counting", it showed strange values. Sometimes sending 0 counts as 1.

Tests: 0: 50 times + 1: 50 times => 50 0: 50 times + 1: 100 times=> 100 0: 100 times + 1: 100 times=> 200 0: 400 times + 1: 20 times => 420 0: 400 times + 1: 1 time => 401

That is, when there are more zeros than ones, then the zeros become one

I continue to test and explore, I will add more information later.

syatihoko commented 9 months ago

Conducted over 500 tests. I will not introduce it here, since it takes a long time to register and there is a lot of garbage. There I also caught UDP losses =) between data centers, so I rule it out.

As I understand it, problems begin with a large number of zeros in the aggregation. But sometimes some kind of magic intervenes =)

1: 10 times + 0: 200 (or 201 or 51...) times => 10 ( 6 times tests) 1: 1 times + 0: 200 times => 1 ( 5 times tests) and 1 time = 201 1: 1 times + 0: 98 to 100 times => 1 ( 6 times tests) 1: 104+ times + 0: 102+ times => 206 ( 6 times tests)

There may be a bug in multimessage.

Config (single node):

n-threads = 8
p-threads = 8
w-threads = 8
a-threads = 7
task-queue-size = 1024
start-as-leader = true
stats-interval = 10000
stats-prefix = "?SECRET?"
consensus = "none"

[carbon]
address = "?SECRET?"
interval = 30000
connect-delay = 250
connect-delay-multiplier = 2
connect-delay-max = 10000
send-retries = 30
chunks = 8

[network]
listen = "0.0.0.0:8126"
peer-listen = "0.0.0.0:8136"
mgmt-listen = "0.0.0.0:8137"
bufsize = 1500
multimessage = true
mm-packets = 1000
mm-async = false
buffer-flush-time = 10000
buffer-flush-length = 655360
greens = 7
async-sockets = 7
nodes = []
snapshot-interval = 1000
[metrics]
log-parse-errors = true

[aggregation]
update-count-threshold = 200

[naming.default]
destination = "smart"
prefix = ""
postfixes = { "min" = "lower", "max" = "upper", "percentile-50" = "percentile.50", "updates" = "" }