Closed reta closed 5 months ago
Benchmark (encoding) (maxBytes) Mode Cnt Score Error Units
AsyncReporterBenchmarks.high_contention JSON 0 thrpt 15 11982053.561 ▒ 1473007.665 ops/s
AsyncReporterBenchmarks.high_contention:messages JSON 0 thrpt 15 1.588 ▒ 3.515 ops/s
AsyncReporterBenchmarks.high_contention:messagesDropped JSON 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.high_contention:spanBacklog JSON 0 thrpt 15 79312.684 ▒ 324.500 ops/s
AsyncReporterBenchmarks.high_contention:spans JSON 0 thrpt 15 95864651.401 ▒ 11787865.324 ops/s
AsyncReporterBenchmarks.high_contention:spansDropped JSON 0 thrpt 15 95794207.879 ▒ 11790659.967 ops/s
AsyncReporterBenchmarks.high_contention JSON 20000000 thrpt 15 3124462.515 ▒ 287800.690 ops/s
AsyncReporterBenchmarks.high_contention:messages JSON 20000000 thrpt 15 755.080 ▒ 16.291 ops/s
AsyncReporterBenchmarks.high_contention:messagesDropped JSON 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.high_contention:spanBacklog JSON 20000000 thrpt 15 79222.183 ▒ 517.571 ops/s
AsyncReporterBenchmarks.high_contention:spans JSON 20000000 thrpt 15 24996956.412 ▒ 2302435.936 ops/s
AsyncReporterBenchmarks.high_contention:spansDropped JSON 20000000 thrpt 15 23200153.194 ▒ 2317497.434 ops/s
AsyncReporterBenchmarks.high_contention THRIFT 0 thrpt 15 9480781.944 ▒ 2559640.133 ops/s
AsyncReporterBenchmarks.high_contention:messages THRIFT 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.high_contention:messagesDropped THRIFT 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.high_contention:spanBacklog THRIFT 0 thrpt 15 79193.912 ▒ 438.878 ops/s
AsyncReporterBenchmarks.high_contention:spans THRIFT 0 thrpt 15 76024899.244 ▒ 20507025.400 ops/s
AsyncReporterBenchmarks.high_contention:spansDropped THRIFT 0 thrpt 15 75964451.094 ▒ 20503731.297 ops/s
AsyncReporterBenchmarks.high_contention THRIFT 20000000 thrpt 15 2025946.139 ▒ 154040.708 ops/s
AsyncReporterBenchmarks.high_contention:messages THRIFT 20000000 thrpt 15 744.973 ▒ 66.512 ops/s
AsyncReporterBenchmarks.high_contention:messagesDropped THRIFT 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.high_contention:spanBacklog THRIFT 20000000 thrpt 15 77403.750 ▒ 5860.366 ops/s
AsyncReporterBenchmarks.high_contention:spans THRIFT 20000000 thrpt 15 16208431.325 ▒ 1232226.253 ops/s
AsyncReporterBenchmarks.high_contention:spansDropped THRIFT 20000000 thrpt 15 14476828.317 ▒ 1113013.712 ops/s
AsyncReporterBenchmarks.high_contention PROTO3 0 thrpt 15 10511120.413 ▒ 1553409.770 ops/s
AsyncReporterBenchmarks.high_contention:messages PROTO3 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.high_contention:messagesDropped PROTO3 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.high_contention:spanBacklog PROTO3 0 thrpt 15 79228.864 ▒ 408.174 ops/s
AsyncReporterBenchmarks.high_contention:spans PROTO3 0 thrpt 15 84160237.979 ▒ 12472649.642 ops/s
AsyncReporterBenchmarks.high_contention:spansDropped PROTO3 0 thrpt 15 84098145.037 ▒ 12479882.673 ops/s
AsyncReporterBenchmarks.high_contention PROTO3 20000000 thrpt 15 2984602.750 ▒ 310582.499 ops/s
AsyncReporterBenchmarks.high_contention:messages PROTO3 20000000 thrpt 15 126.424 ▒ 13.678 ops/s
AsyncReporterBenchmarks.high_contention:messagesDropped PROTO3 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.high_contention:spanBacklog PROTO3 20000000 thrpt 15 79352.144 ▒ 393.599 ops/s
AsyncReporterBenchmarks.high_contention:spans PROTO3 20000000 thrpt 15 23878565.592 ▒ 2484506.262 ops/s
AsyncReporterBenchmarks.high_contention:spansDropped PROTO3 20000000 thrpt 15 23039360.871 ▒ 2422731.629 ops/s
AsyncReporterBenchmarks.mild_contention JSON 0 thrpt 15 6567318.331 ▒ 743675.929 ops/s
AsyncReporterBenchmarks.mild_contention:messages JSON 0 thrpt 15 0.396 ▒ 0.876 ops/s
AsyncReporterBenchmarks.mild_contention:messagesDropped JSON 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.mild_contention:spanBacklog JSON 0 thrpt 15 19816.363 ▒ 125.625 ops/s
AsyncReporterBenchmarks.mild_contention:spans JSON 0 thrpt 15 13134741.031 ▒ 1487321.671 ops/s
AsyncReporterBenchmarks.mild_contention:spansDropped JSON 0 thrpt 15 13116184.864 ▒ 1488299.198 ops/s
AsyncReporterBenchmarks.mild_contention JSON 20000000 thrpt 15 3861378.895 ▒ 243729.117 ops/s
AsyncReporterBenchmarks.mild_contention:messages JSON 20000000 thrpt 15 197.226 ▒ 9.210 ops/s
AsyncReporterBenchmarks.mild_contention:messagesDropped JSON 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.mild_contention:spanBacklog JSON 20000000 thrpt 15 19827.952 ▒ 120.095 ops/s
AsyncReporterBenchmarks.mild_contention:spans JSON 20000000 thrpt 15 7722772.516 ▒ 487454.333 ops/s
AsyncReporterBenchmarks.mild_contention:spansDropped JSON 20000000 thrpt 15 7253910.768 ▒ 473705.432 ops/s
AsyncReporterBenchmarks.mild_contention THRIFT 0 thrpt 15 5146045.967 ▒ 1386333.962 ops/s
AsyncReporterBenchmarks.mild_contention:messages THRIFT 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.mild_contention:messagesDropped THRIFT 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.mild_contention:spanBacklog THRIFT 0 thrpt 15 19796.501 ▒ 69.635 ops/s
AsyncReporterBenchmarks.mild_contention:spans THRIFT 0 thrpt 15 10293587.808 ▒ 2772247.834 ops/s
AsyncReporterBenchmarks.mild_contention:spansDropped THRIFT 0 thrpt 15 10274125.487 ▒ 2772226.463 ops/s
AsyncReporterBenchmarks.mild_contention THRIFT 20000000 thrpt 15 2153647.544 ▒ 242074.188 ops/s
AsyncReporterBenchmarks.mild_contention:messages THRIFT 20000000 thrpt 15 232.573 ▒ 22.177 ops/s
AsyncReporterBenchmarks.mild_contention:messagesDropped THRIFT 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.mild_contention:spanBacklog THRIFT 20000000 thrpt 15 19549.955 ▒ 1207.824 ops/s
AsyncReporterBenchmarks.mild_contention:spans THRIFT 20000000 thrpt 15 4307301.325 ▒ 484147.901 ops/s
AsyncReporterBenchmarks.mild_contention:spansDropped THRIFT 20000000 thrpt 15 3770981.590 ▒ 437394.511 ops/s
AsyncReporterBenchmarks.mild_contention PROTO3 0 thrpt 15 5549081.224 ▒ 821283.678 ops/s
AsyncReporterBenchmarks.mild_contention:messages PROTO3 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.mild_contention:messagesDropped PROTO3 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.mild_contention:spanBacklog PROTO3 0 thrpt 15 19774.955 ▒ 78.588 ops/s
AsyncReporterBenchmarks.mild_contention:spans PROTO3 0 thrpt 15 11106110.496 ▒ 1650945.934 ops/s
AsyncReporterBenchmarks.mild_contention:spansDropped PROTO3 0 thrpt 15 11088133.705 ▒ 1651723.019 ops/s
AsyncReporterBenchmarks.mild_contention PROTO3 20000000 thrpt 15 3638809.421 ▒ 896143.684 ops/s
AsyncReporterBenchmarks.mild_contention:messages PROTO3 20000000 thrpt 15 35.507 ▒ 8.528 ops/s
AsyncReporterBenchmarks.mild_contention:messagesDropped PROTO3 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.mild_contention:spanBacklog PROTO3 20000000 thrpt 15 19799.094 ▒ 81.589 ops/s
AsyncReporterBenchmarks.mild_contention:spans PROTO3 20000000 thrpt 15 7277633.629 ▒ 1792282.234 ops/s
AsyncReporterBenchmarks.mild_contention:spansDropped PROTO3 20000000 thrpt 15 7043998.063 ▒ 1747169.364 ops/s
AsyncReporterBenchmarks.no_contention JSON 0 thrpt 15 18211292.933 ▒ 4190088.557 ops/s
AsyncReporterBenchmarks.no_contention:messages JSON 0 thrpt 15 0.198 ▒ 0.438 ops/s
AsyncReporterBenchmarks.no_contention:messagesDropped JSON 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.no_contention:spanBacklog JSON 0 thrpt 15 9896.842 ▒ 33.616 ops/s
AsyncReporterBenchmarks.no_contention:spans JSON 0 thrpt 15 18211292.933 ▒ 4190088.557 ops/s
AsyncReporterBenchmarks.no_contention:spansDropped JSON 0 thrpt 15 18200945.852 ▒ 4190357.391 ops/s
AsyncReporterBenchmarks.no_contention JSON 20000000 thrpt 15 4485115.872 ▒ 74791.479 ops/s
AsyncReporterBenchmarks.no_contention:messages JSON 20000000 thrpt 15 105.892 ▒ 3.077 ops/s
AsyncReporterBenchmarks.no_contention:messagesDropped JSON 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.no_contention:spanBacklog JSON 20000000 thrpt 15 9781.858 ▒ 518.319 ops/s
AsyncReporterBenchmarks.no_contention:spans JSON 20000000 thrpt 15 4485115.872 ▒ 74791.479 ops/s
AsyncReporterBenchmarks.no_contention:spansDropped JSON 20000000 thrpt 15 4234217.808 ▒ 75548.455 ops/s
AsyncReporterBenchmarks.no_contention THRIFT 0 thrpt 15 9502824.461 ▒ 3209696.754 ops/s
AsyncReporterBenchmarks.no_contention:messages THRIFT 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.no_contention:messagesDropped THRIFT 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.no_contention:spanBacklog THRIFT 0 thrpt 15 9906.573 ▒ 50.874 ops/s
AsyncReporterBenchmarks.no_contention:spans THRIFT 0 thrpt 15 9502824.461 ▒ 3209696.754 ops/s
AsyncReporterBenchmarks.no_contention:spansDropped THRIFT 0 thrpt 15 9492917.888 ▒ 3209693.647 ops/s
AsyncReporterBenchmarks.no_contention THRIFT 20000000 thrpt 15 1533220.272 ▒ 40817.118 ops/s
AsyncReporterBenchmarks.no_contention:messages THRIFT 20000000 thrpt 15 128.154 ▒ 3.384 ops/s
AsyncReporterBenchmarks.no_contention:messagesDropped THRIFT 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.no_contention:spanBacklog THRIFT 20000000 thrpt 15 9504.011 ▒ 741.079 ops/s
AsyncReporterBenchmarks.no_contention:spans THRIFT 20000000 thrpt 15 1533220.272 ▒ 40817.118 ops/s
AsyncReporterBenchmarks.no_contention:spansDropped THRIFT 20000000 thrpt 15 1238957.988 ▒ 33520.382 ops/s
AsyncReporterBenchmarks.no_contention PROTO3 0 thrpt 15 11498394.230 ▒ 899814.975 ops/s
AsyncReporterBenchmarks.no_contention:messages PROTO3 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.no_contention:messagesDropped PROTO3 0 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.no_contention:spanBacklog PROTO3 0 thrpt 15 9918.370 ▒ 62.796 ops/s
AsyncReporterBenchmarks.no_contention:spans PROTO3 0 thrpt 15 11498394.230 ▒ 899814.975 ops/s
AsyncReporterBenchmarks.no_contention:spansDropped PROTO3 0 thrpt 15 11488475.860 ▒ 899821.160 ops/s
AsyncReporterBenchmarks.no_contention PROTO3 20000000 thrpt 15 4111289.711 ▒ 46384.507 ops/s
AsyncReporterBenchmarks.no_contention:messages PROTO3 20000000 thrpt 15 20.474 ▒ 1.317 ops/s
AsyncReporterBenchmarks.no_contention:messagesDropped PROTO3 20000000 thrpt 15 ? 0 ops/s
AsyncReporterBenchmarks.no_contention:spanBacklog PROTO3 20000000 thrpt 15 9908.434 ▒ 64.619 ops/s
AsyncReporterBenchmarks.no_contention:spans PROTO3 20000000 thrpt 15 4111289.711 ▒ 46384.507 ops/s
AsyncReporterBenchmarks.no_contention:spansDropped PROTO3 20000000 thrpt 15 3978043.429 ▒ 51157.232 ops/s
So, to restate assumptions here. We are allowing queuedMaxBytes=0 as a way to avoid sizing the span (microsecond, could be much less) in the critical path.
What maybe I wasn't clear about is that in a library as well used as this is, we really cannot rationalize removing checks about the safety of outbound messages.
We should also not interfere with metrics except the one which is about the current size of the queue in bytes, which is no longer possible to calculate. So, this means, the behaviour before and after should be the same otherwise, same metrics, same drop behavior on send.. just there may be a case where we enqueue something too big, and drop it later (on the flush/reporter thread).
Let's do it right or don't do it. I already spent a lot of time with you on this, so can help contribute to your branch, if you are running out of steam. I'd prefer us not bring this issue into another week, if we can avoid it.
Let's do it right or don't do it. I already spent a lot of time with you on this, so can help contribute to your branch, if you are running out of steam. I'd prefer us not bring this issue into another week, if we can avoid it.
Please feel free to push the changes (I am not running out of steam but there is limited time I have to dedicate working on this), I am planning to finalize the tests for AsyncReporterTest + queuedMaxBytes={0, 1000000}
tomorrow so we could make sure all metrics are captured as it should.
I created a diff on another branch about the core code change I think is needed. Basically, this pulls the (count) bounded queue into an abstract class, used directly when the size bound is zero. It then overrides behaviour when there is a size bound.
There is tension around guarding on message size. Instead of passing a lot of stuff to the queue types, I chose to use an anonymous subclass inside async reporter, as then it already has all the objects needed. you can decide to factor it otherwise, but I think it is picking poison anyway. this is just one type of poison which I think results in the least code to maintain. check me!
Note: all your other fixes around bench being broken etc are still needed.. this was just a sketch to get us to a point where we still guard on size regardless of which thread is doing it.
https://github.com/openzipkin/zipkin-reporter-java/compare/less-code
Note: all your other fixes around bench being broken etc are still needed.. this was just a sketch to get us to a point where we still guard on size regardless of which thread is doing it.
Thanks for that 🥇 , I kept 2 classes (it is easier to test in isolation) and CountBoundedQueue
fully respects metrics and messageMaxBytes
, thanks for pointing the gap here. I've added bunch of tests (parameterize large chunk of existing AsyncReporterTest
), please feel free to go with "less code" approach, the tests should have your back now,
ps after this change and before the next minor, we should default to queuedMaxBytes 0 I think.
In doing that I think we can close #204 because we won't need to calculate 1pct of memory anymore.
As a side note we can consider removing the memory bound on the next major version of reporter as I think it causes more problems than it solves in hindsight. That said, per #247 we wouldn't soon remove it.. just possibly we can deprecate the builder functions for removal in 4 if you agree.
above I meant deprecating for removal setting queuedMaxBytes > 0. the other bounds still make sense, like max size per message.
oops sorry the other PR drifted you as you had fixed benchmarks here 😿
@codefromthecrypt one last look? :pray:
Branched off
SizeBoundedQueue
out ofByteBoundedQueue
for cases whenqueuedMaxBytes=0
.