openzipkin / brave

Java distributed tracing implementation compatible with Zipkin backend services.
Apache License 2.0
2.36k stars 713 forks source link

Fix VirtualThread pinning with RealSpan and zipkin2.reporter by switching from synchronized to ReentrantLock #1427

Closed reta closed 7 months ago

reta commented 8 months ago

Closes https://github.com/openzipkin/brave/issues/1426

TODO:

reta commented 8 months ago

Rerun tracing benchmarks on current branch (master), java -jar instrumentation/benchmarks/target/benchmarks.jar TracerBenchmarks:

Benchmark                                                                  Mode     Cnt      Score   Error  Units
TracerBenchmarks.currentSpan                                             sample  775994      0.061 ± 0.002  us/op
TracerBenchmarks.currentSpan:p0.00                                       sample                ≈ 0          us/op
TracerBenchmarks.currentSpan:p0.50                                       sample              0.100          us/op
TracerBenchmarks.currentSpan:p0.90                                       sample              0.100          us/op
TracerBenchmarks.currentSpan:p0.95                                       sample              0.100          us/op
TracerBenchmarks.currentSpan:p0.99                                       sample              0.100          us/op
TracerBenchmarks.currentSpan:p0.999                                      sample              0.200          us/op
TracerBenchmarks.currentSpan:p0.9999                                     sample             17.888          us/op
TracerBenchmarks.currentSpan:p1.00                                       sample            281.600          us/op
TracerBenchmarks.currentSpan_tag                                         sample  790450      0.224 ± 0.018  us/op
TracerBenchmarks.currentSpan_tag:p0.00                                   sample                ≈ 0          us/op
TracerBenchmarks.currentSpan_tag:p0.50                                   sample              0.200          us/op
TracerBenchmarks.currentSpan_tag:p0.90                                   sample              0.300          us/op
TracerBenchmarks.currentSpan_tag:p0.95                                   sample              0.300          us/op
TracerBenchmarks.currentSpan_tag:p0.99                                   sample              0.800          us/op
TracerBenchmarks.currentSpan_tag:p0.999                                  sample             10.896          us/op
TracerBenchmarks.currentSpan_tag:p0.9999                                 sample             27.484          us/op
TracerBenchmarks.currentSpan_tag:p1.00                                   sample           1849.344          us/op
TracerBenchmarks.currentSpan_unsampled                                   sample  794814      0.061 ± 0.001  us/op
TracerBenchmarks.currentSpan_unsampled:p0.00                             sample                ≈ 0          us/op
TracerBenchmarks.currentSpan_unsampled:p0.50                             sample              0.100          us/op
TracerBenchmarks.currentSpan_unsampled:p0.90                             sample              0.100          us/op
TracerBenchmarks.currentSpan_unsampled:p0.95                             sample              0.100          us/op
TracerBenchmarks.currentSpan_unsampled:p0.99                             sample              0.100          us/op
TracerBenchmarks.currentSpan_unsampled:p0.999                            sample              0.200          us/op
TracerBenchmarks.currentSpan_unsampled:p0.9999                           sample             17.984          us/op
TracerBenchmarks.currentSpan_unsampled:p1.00                             sample            112.384          us/op
TracerBenchmarks.joinWithSpanInScope                                     sample  932503      1.157 ± 0.062  us/op
TracerBenchmarks.joinWithSpanInScope:p0.00                               sample              0.299          us/op
TracerBenchmarks.joinWithSpanInScope:p0.50                               sample              0.800          us/op
TracerBenchmarks.joinWithSpanInScope:p0.90                               sample              1.500          us/op
TracerBenchmarks.joinWithSpanInScope:p0.95                               sample              1.700          us/op
TracerBenchmarks.joinWithSpanInScope:p0.99                               sample              3.600          us/op
TracerBenchmarks.joinWithSpanInScope:p0.999                              sample             26.880          us/op
TracerBenchmarks.joinWithSpanInScope:p0.9999                             sample            102.208          us/op
TracerBenchmarks.joinWithSpanInScope:p1.00                               sample          10584.064          us/op
TracerBenchmarks.joinWithSpanInScope_baggage                             sample  865058      1.284 ± 0.075  us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.00                       sample              0.300          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.50                       sample              0.900          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.90                       sample              1.600          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.95                       sample              1.900          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.99                       sample              4.800          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.999                      sample             32.960          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.9999                     sample            102.525          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p1.00                       sample          12173.312          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal                        sample  951963      1.151 ± 0.073  us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.00                  sample              0.300          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.50                  sample              0.800          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.90                  sample              1.400          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.95                  sample              1.700          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.99                  sample              3.200          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.999                 sample             26.880          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.9999                sample            102.942          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p1.00                  sample          14450.688          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage                sample  889799      1.431 ± 0.071  us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.00          sample              0.200          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.50          sample              0.900          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.90          sample              1.800          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.95          sample              2.300          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.99          sample              6.696          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.999         sample             48.000          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.9999        sample            140.032          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p1.00          sample           6144.000          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled                           sample  815324      0.129 ± 0.021  us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.00                     sample                ≈ 0          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.50                     sample              0.100          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.90                     sample              0.100          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.95                     sample              0.200          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.99                     sample              0.200          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.999                    sample              9.488          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.9999                   sample             24.992          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p1.00                     sample           2863.104          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage                   sample  729570      0.253 ± 0.027  us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.00             sample                ≈ 0          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.50             sample              0.200          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.90             sample              0.300          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.95             sample              0.400          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.99             sample              0.800          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.999            sample             16.000          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.9999           sample             34.376          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p1.00             sample           2666.496          us/op
TracerBenchmarks.newChildWithSpanInScope                                 sample  688420      0.826 ± 0.053  us/op
TracerBenchmarks.newChildWithSpanInScope:p0.00                           sample              0.399          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.50                           sample              0.600          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.90                           sample              0.900          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.95                           sample              1.000          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.99                           sample              1.300          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.999                          sample             19.488          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.9999                         sample             68.386          us/op
TracerBenchmarks.newChildWithSpanInScope:p1.00                           sample           3301.376          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage                         sample  843103      0.938 ± 0.042  us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.00                   sample              0.400          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.50                   sample              0.800          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.90                   sample              1.000          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.95                   sample              1.100          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.99                   sample              1.500          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.999                  sample             19.200          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.9999                 sample             57.132          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p1.00                   sample           2596.864          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal                    sample  672946      0.828 ± 0.049  us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.00              sample              0.400          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.50              sample              0.600          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.90              sample              0.900          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.95              sample              1.000          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.99              sample              1.400          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.999             sample             19.296          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.9999            sample             51.068          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p1.00              sample           3178.496          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage            sample  934521      0.926 ± 0.037  us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.00      sample              0.400          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.50      sample              0.800          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.90      sample              1.100          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.95      sample              1.200          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.99      sample              1.300          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.999     sample             18.688          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.9999    sample             54.924          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p1.00      sample           3756.032          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled                       sample  953007      0.083 ± 0.018  us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.00                 sample                ≈ 0          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.50                 sample              0.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.90                 sample              0.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.95                 sample              0.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.99                 sample              0.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.999                sample              1.099          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.9999               sample             19.072          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p1.00                 sample           2867.200          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage               sample  834183      0.247 ± 0.026  us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.00         sample                ≈ 0          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.50         sample              0.200          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.90         sample              0.300          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.95         sample              0.300          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.99         sample              0.400          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.999        sample             15.678          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.9999       sample             36.618          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p1.00         sample           3059.712          us/op
TracerBenchmarks.nextWithSpanInScope                                     sample  699486      0.782 ± 0.043  us/op
TracerBenchmarks.nextWithSpanInScope:p0.00                               sample              0.400          us/op
TracerBenchmarks.nextWithSpanInScope:p0.50                               sample              0.600          us/op
TracerBenchmarks.nextWithSpanInScope:p0.90                               sample              0.800          us/op
TracerBenchmarks.nextWithSpanInScope:p0.95                               sample              1.000          us/op
TracerBenchmarks.nextWithSpanInScope:p0.99                               sample              1.200          us/op
TracerBenchmarks.nextWithSpanInScope:p0.999                              sample             18.496          us/op
TracerBenchmarks.nextWithSpanInScope:p0.9999                             sample             54.625          us/op
TracerBenchmarks.nextWithSpanInScope:p1.00                               sample           3108.864          us/op
TracerBenchmarks.nextWithSpanInScope_baggage                             sample  912570      0.923 ± 0.037  us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.00                       sample              0.400          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.50                       sample              0.800          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.90                       sample              1.100          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.95                       sample              1.100          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.99                       sample              1.500          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.999                      sample             18.688          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.9999                     sample             51.085          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p1.00                       sample           3600.384          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled                           sample  971978      0.108 ± 0.002  us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.00                     sample                ≈ 0          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.50                     sample              0.100          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.90                     sample              0.200          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.95                     sample              0.200          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.99                     sample              0.200          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.999                    sample              3.800          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.9999                   sample             22.496          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p1.00                     sample            155.648          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage                   sample  807001      0.282 ± 0.043  us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.00             sample              0.099          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.50             sample              0.200          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.90             sample              0.300          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.95             sample              0.300          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.99             sample              0.400          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.999            sample             12.992          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.9999           sample             32.538          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p1.00             sample           3428.352          us/op
TracerBenchmarks.startScopedSpanWithParent                               sample  806921      0.696 ± 0.046  us/op
TracerBenchmarks.startScopedSpanWithParent:p0.00                         sample              0.300          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.50                         sample              0.500          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.90                         sample              0.700          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.95                         sample              0.800          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.99                         sample              1.000          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.999                        sample             17.991          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.9999                       sample             41.339          us/op
TracerBenchmarks.startScopedSpanWithParent:p1.00                         sample           3317.760          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage                       sample  692882      0.832 ± 0.056  us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.00                 sample              0.300          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.50                 sample              0.600          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.90                 sample              0.800          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.95                 sample              0.900          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.99                 sample              1.200          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.999                sample             19.776          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.9999               sample             76.854          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p1.00                 sample           2965.504          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal                  sample  775959      0.735 ± 0.046  us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.00            sample              0.300          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.50            sample              0.600          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.90            sample              0.800          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.95            sample              0.900          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.99            sample              1.100          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.999           sample             18.080          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.9999          sample             66.170          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p1.00            sample           2846.720          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage          sample  663385      0.832 ± 0.043  us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.00    sample              0.300          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.50    sample              0.700          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.90    sample              0.900          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.95    sample              1.000          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.99    sample              1.400          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.999   sample             22.400          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.9999  sample             74.577          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p1.00    sample           3436.544          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled                     sample  915326      0.111 ± 0.012  us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.00               sample                ≈ 0          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.50               sample              0.100          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.90               sample              0.100          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.95               sample              0.200          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.99               sample              0.200          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.999              sample              7.265          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.9999             sample             22.880          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p1.00               sample           2347.008          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage             sample  814910      0.247 ± 0.035  us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.00       sample                ≈ 0          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.50       sample              0.200          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.90       sample              0.300          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.95       sample              0.300          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.99       sample              0.400          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.999      sample             14.592          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.9999     sample             31.749          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p1.00       sample           3772.416          us/op
reta commented 8 months ago

On this change (java -jar instrumentation/benchmarks/target/benchmarks.jar TracerBenchmarks):


Benchmark                                                                  Mode     Cnt     Score   Error  Units
TracerBenchmarks.currentSpan                                             sample  784816     0.063 ± 0.002  us/op
TracerBenchmarks.currentSpan:p0.00                                       sample               ≈ 0          us/op
TracerBenchmarks.currentSpan:p0.50                                       sample             0.100          us/op
TracerBenchmarks.currentSpan:p0.90                                       sample             0.100          us/op
TracerBenchmarks.currentSpan:p0.95                                       sample             0.100          us/op
TracerBenchmarks.currentSpan:p0.99                                       sample             0.100          us/op
TracerBenchmarks.currentSpan:p0.999                                      sample             0.200          us/op
TracerBenchmarks.currentSpan:p0.9999                                     sample            18.834          us/op
TracerBenchmarks.currentSpan:p1.00                                       sample           124.160          us/op
TracerBenchmarks.currentSpan_tag                                         sample  763116     0.367 ± 0.029  us/op
TracerBenchmarks.currentSpan_tag:p0.00                                   sample               ≈ 0          us/op
TracerBenchmarks.currentSpan_tag:p0.50                                   sample             0.200          us/op
TracerBenchmarks.currentSpan_tag:p0.90                                   sample             0.500          us/op
TracerBenchmarks.currentSpan_tag:p0.95                                   sample             0.600          us/op
TracerBenchmarks.currentSpan_tag:p0.99                                   sample             1.100          us/op
TracerBenchmarks.currentSpan_tag:p0.999                                  sample            23.584          us/op
TracerBenchmarks.currentSpan_tag:p0.9999                                 sample            66.608          us/op
TracerBenchmarks.currentSpan_tag:p1.00                                   sample          4734.976          us/op
TracerBenchmarks.currentSpan_unsampled                                   sample  785412     0.093 ± 0.004  us/op
TracerBenchmarks.currentSpan_unsampled:p0.00                             sample               ≈ 0          us/op
TracerBenchmarks.currentSpan_unsampled:p0.50                             sample             0.100          us/op
TracerBenchmarks.currentSpan_unsampled:p0.90                             sample             0.100          us/op
TracerBenchmarks.currentSpan_unsampled:p0.95                             sample             0.100          us/op
TracerBenchmarks.currentSpan_unsampled:p0.99                             sample             0.200          us/op
TracerBenchmarks.currentSpan_unsampled:p0.999                            sample            11.488          us/op
TracerBenchmarks.currentSpan_unsampled:p0.9999                           sample            35.544          us/op
TracerBenchmarks.currentSpan_unsampled:p1.00                             sample           260.352          us/op
TracerBenchmarks.joinWithSpanInScope                                     sample  932652     1.152 ± 0.039  us/op
TracerBenchmarks.joinWithSpanInScope:p0.00                               sample             0.300          us/op
TracerBenchmarks.joinWithSpanInScope:p0.50                               sample             0.800          us/op
TracerBenchmarks.joinWithSpanInScope:p0.90                               sample             1.400          us/op
TracerBenchmarks.joinWithSpanInScope:p0.95                               sample             1.700          us/op
TracerBenchmarks.joinWithSpanInScope:p0.99                               sample             3.900          us/op
TracerBenchmarks.joinWithSpanInScope:p0.999                              sample            36.588          us/op
TracerBenchmarks.joinWithSpanInScope:p0.9999                             sample           120.568          us/op
TracerBenchmarks.joinWithSpanInScope:p1.00                               sample          3506.176          us/op
TracerBenchmarks.joinWithSpanInScope_baggage                             sample  789384     1.567 ± 0.064  us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.00                       sample             0.300          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.50                       sample             1.100          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.90                       sample             1.900          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.95                       sample             2.300          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.99                       sample             8.496          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.999                      sample            53.888          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p0.9999                     sample           154.703          us/op
TracerBenchmarks.joinWithSpanInScope_baggage:p1.00                       sample          6086.656          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal                        sample  904546     1.171 ± 0.039  us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.00                  sample             0.300          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.50                  sample             0.800          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.90                  sample             1.400          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.95                  sample             1.700          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.99                  sample             3.900          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.999                 sample            37.184          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.9999                sample           120.134          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal:p1.00                  sample          5726.208          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage                sample  796288     2.603 ± 0.096  us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.00          sample             0.400          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.50          sample             1.700          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.90          sample             3.200          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.95          sample             3.900          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.99          sample            21.696          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.999         sample            93.147          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.9999        sample           324.460          us/op
TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p1.00          sample          9240.576          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled                           sample  897342     0.152 ± 0.006  us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.00                     sample               ≈ 0          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.50                     sample             0.100          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.90                     sample             0.200          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.95                     sample             0.200          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.99                     sample             0.400          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.999                    sample            16.192          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p0.9999                   sample            38.848          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled:p1.00                     sample          1398.784          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage                   sample  876918     0.407 ± 0.030  us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.00             sample               ≈ 0          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.50             sample             0.300          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.90             sample             0.500          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.95             sample             0.600          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.99             sample             1.100          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.999            sample            22.696          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.9999           sample            87.975          us/op
TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p1.00             sample          3280.896          us/op
TracerBenchmarks.newChildWithSpanInScope                                 sample  702916     0.916 ± 0.051  us/op
TracerBenchmarks.newChildWithSpanInScope:p0.00                           sample             0.400          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.50                           sample             0.700          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.90                           sample             1.000          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.95                           sample             1.100          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.99                           sample             1.700          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.999                          sample            23.680          us/op
TracerBenchmarks.newChildWithSpanInScope:p0.9999                         sample            77.339          us/op
TracerBenchmarks.newChildWithSpanInScope:p1.00                           sample          3284.992          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage                         sample  794588     1.573 ± 0.073  us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.00                   sample             0.499          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.50                   sample             1.100          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.90                   sample             1.800          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.95                   sample             2.100          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.99                   sample             5.000          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.999                  sample            50.240          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p0.9999                 sample           236.565          us/op
TracerBenchmarks.newChildWithSpanInScope_baggage:p1.00                   sample          4603.904          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal                    sample  783725     0.945 ± 0.042  us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.00              sample             0.400          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.50              sample             0.700          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.90              sample             1.100          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.95              sample             1.200          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.99              sample             1.900          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.999             sample            25.184          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.9999            sample            75.216          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p1.00              sample          2465.792          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage            sample  830123     1.368 ± 0.072  us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.00      sample             0.400          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.50      sample             1.000          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.90      sample             1.500          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.95      sample             1.800          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.99      sample             3.500          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.999     sample            40.768          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.9999    sample           132.096          us/op
TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p1.00      sample          7651.328          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled                       sample  897578     0.109 ± 0.015  us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.00                 sample               ≈ 0          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.50                 sample             0.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.90                 sample             0.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.95                 sample             0.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.99                 sample             0.300          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.999                sample            11.888          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.9999               sample            37.502          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled:p1.00                 sample          3952.640          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage               sample  810389     0.550 ± 0.055  us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.00         sample             0.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.50         sample             0.300          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.90         sample             0.600          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.95         sample             0.700          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.99         sample             1.100          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.999        sample            31.680          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.9999       sample           126.459          us/op
TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p1.00         sample          7184.384          us/op
TracerBenchmarks.nextWithSpanInScope                                     sample  870070     1.012 ± 0.062  us/op
TracerBenchmarks.nextWithSpanInScope:p0.00                               sample             0.400          us/op
TracerBenchmarks.nextWithSpanInScope:p0.50                               sample             0.700          us/op
TracerBenchmarks.nextWithSpanInScope:p0.90                               sample             1.100          us/op
TracerBenchmarks.nextWithSpanInScope:p0.95                               sample             1.300          us/op
TracerBenchmarks.nextWithSpanInScope:p0.99                               sample             2.000          us/op
TracerBenchmarks.nextWithSpanInScope:p0.999                              sample            30.272          us/op
TracerBenchmarks.nextWithSpanInScope:p0.9999                             sample           101.631          us/op
TracerBenchmarks.nextWithSpanInScope:p1.00                               sample          5464.064          us/op
TracerBenchmarks.nextWithSpanInScope_baggage                             sample  833799     1.365 ± 0.064  us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.00                       sample             0.500          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.50                       sample             1.000          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.90                       sample             1.500          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.95                       sample             1.800          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.99                       sample             3.200          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.999                      sample            40.768          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p0.9999                     sample           153.564          us/op
TracerBenchmarks.nextWithSpanInScope_baggage:p1.00                       sample          5144.576          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled                           sample  776152     0.127 ± 0.005  us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.00                     sample               ≈ 0          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.50                     sample             0.100          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.90                     sample             0.200          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.95                     sample             0.200          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.99                     sample             0.300          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.999                    sample            15.200          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p0.9999                   sample            41.984          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled:p1.00                     sample           449.024          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage                   sample  964087     0.412 ± 0.044  us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.00             sample             0.100          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.50             sample             0.200          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.90             sample             0.500          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.95             sample             0.600          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.99             sample             1.000          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.999            sample            24.192          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.9999           sample            84.399          us/op
TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p1.00             sample          5873.664          us/op
TracerBenchmarks.startScopedSpanWithParent                               sample  838624     1.380 ± 0.075  us/op
TracerBenchmarks.startScopedSpanWithParent:p0.00                         sample             0.300          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.50                         sample             0.900          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.90                         sample             1.700          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.95                         sample             2.000          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.99                         sample             3.600          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.999                        sample            48.960          us/op
TracerBenchmarks.startScopedSpanWithParent:p0.9999                       sample           161.280          us/op
TracerBenchmarks.startScopedSpanWithParent:p1.00                         sample          7872.512          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage                       sample  968190     1.005 ± 0.052  us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.00                 sample             0.400          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.50                 sample             0.800          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.90                 sample             1.100          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.95                 sample             1.300          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.99                 sample             1.900          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.999                sample            26.592          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p0.9999               sample            96.430          us/op
TracerBenchmarks.startScopedSpanWithParent_baggage:p1.00                 sample          5013.504          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal                  sample  871479     1.302 ± 0.089  us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.00            sample             0.300          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.50            sample             0.900          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.90            sample             1.500          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.95            sample             1.800          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.99            sample             3.000          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.999           sample            40.768          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.9999          sample           124.397          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p1.00            sample          8388.608          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage          sample  872578     1.160 ± 0.059  us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.00    sample             0.400          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.50    sample             0.800          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.90    sample             1.300          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.95    sample             1.600          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.99    sample             2.700          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.999   sample            36.544          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.9999  sample           155.772          us/op
TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p1.00    sample          4562.944          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled                     sample  873649     0.167 ± 0.024  us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.00               sample               ≈ 0          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.50               sample             0.100          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.90               sample             0.200          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.95               sample             0.200          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.99               sample             0.300          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.999              sample            19.234          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.9999             sample            53.584          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled:p1.00               sample          4194.304          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage             sample  961982     0.405 ± 0.037  us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.00       sample               ≈ 0          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.50       sample             0.300          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.90       sample             0.400          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.95       sample             0.500          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.99       sample             0.800          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.999      sample            21.888          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.9999     sample            77.645          us/op
TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p1.00       sample          5824.512          us/op```
reta commented 8 months ago
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Benchmark                                                                 Mode   |    Cnt |     Score   Error |    Cnt |     Score  Error |    Cnt    |    Score   Error |     Cnt |    Score   Error | Units | 
|                                                                                  | (main) |                   |   (pr) |                  | (main,vt) |                  | (pr,vt) |                  |       |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| TracerBenchmarks.currentSpan                                            | sample | 775994 |     0.061 ± 0.002 | 784816 |    0.063 ± 0.002 |    744101 |    0.081 ± 0.003 |  765039 |    0.067 ± 0.007 | us/op |
| TracerBenchmarks.currentSpan:p0.00                                      | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.currentSpan:p0.50                                      | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.currentSpan:p0.90                                      | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.currentSpan:p0.95                                      | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.currentSpan:p0.99                                      | sample |        |     0.100         |        |    0.100         |           |    0.101         |         |    0.100         | us/op |
| TracerBenchmarks.currentSpan:p0.999                                     | sample |        |     0.200         |        |    0.200         |           |    8.096         |         |    0.500         | us/op |
| TracerBenchmarks.currentSpan:p0.9999                                    | sample |        |    17.888         |        |   18.834         |           |   27.409         |         |   24.096         | us/op |
| TracerBenchmarks.currentSpan:p1.00                                      | sample |        |   281.600         |        |  124.160         |           |  234.752         |         | 1390.592         | us/op |
| TracerBenchmarks.currentSpan_tag                                        | sample | 790450 |     0.224 ± 0.018 | 763116 |    0.367 ± 0.029 |    726105 |    0.258 ± 0.022 |  915827 |    0.318 ± 0.023 | us/op |
| TracerBenchmarks.currentSpan_tag:p0.00                                  | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.currentSpan_tag:p0.50                                  | sample |        |     0.200         |        |    0.200         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.currentSpan_tag:p0.90                                  | sample |        |     0.300         |        |    0.500         |           |    0.300         |         |    0.400         | us/op |
| TracerBenchmarks.currentSpan_tag:p0.95                                  | sample |        |     0.300         |        |    0.600         |           |    0.400         |         |    0.500         | us/op |
| TracerBenchmarks.currentSpan_tag:p0.99                                  | sample |        |     0.800         |        |    1.100         |           |    1.000         |         |    1.400         | us/op |
| TracerBenchmarks.currentSpan_tag:p0.999                                 | sample |        |    10.896         |        |   23.584         |           |   16.672         |         |   18.080         | us/op |
| TracerBenchmarks.currentSpan_tag:p0.9999                                | sample |        |    27.484         |        |   66.608         |           |   33.152         |         |   48.000         | us/op |
| TracerBenchmarks.currentSpan_tag:p1.00                                  | sample |        |  1849.344         |        | 4734.976         |           | 2297.856         |         | 3014.656         | us/op |
| TracerBenchmarks.currentSpan_unsampled                                  | sample | 794814 |     0.061 ± 0.001 | 785412 |    0.093 ± 0.004 |    774883 |    0.066 ± 0.002 |  758848 |    0.065 ± 0.003 | us/op |
| TracerBenchmarks.currentSpan_unsampled:p0.00                            | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.currentSpan_unsampled:p0.50                            | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.currentSpan_unsampled:p0.90                            | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.currentSpan_unsampled:p0.95                            | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.currentSpan_unsampled:p0.99                            | sample |        |     0.100         |        |    0.200         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.currentSpan_unsampled:p0.999                           | sample |        |     0.200         |        |   11.488         |           |    0.400         |         |    0.500         | us/op |
| TracerBenchmarks.currentSpan_unsampled:p0.9999                          | sample |        |    17.984         |        |   35.544         |           |   22.176         |         |   21.217         | us/op |
| TracerBenchmarks.currentSpan_unsampled:p1.00                            | sample |        |   112.384         |        |  260.352         |           |  171.264         |         |  409.088         | us/op |
| TracerBenchmarks.joinWithSpanInScope                                    | sample | 932503 |     1.157 ± 0.062 | 932652 |    1.152 ± 0.039 |    807669 |    1.487 ± 0.063 |  786776 |    1.494 ± 0.063 | us/op |
| TracerBenchmarks.joinWithSpanInScope:p0.00                              | sample |        |     0.299         |        |    0.300         |           |    0.300         |         |    0.300         | us/op |
| TracerBenchmarks.joinWithSpanInScope:p0.50                              | sample |        |     0.800         |        |    0.800         |           |    1.000         |         |    1.000         | us/op |
| TracerBenchmarks.joinWithSpanInScope:p0.90                              | sample |        |     1.500         |        |    1.400         |           |    1.800         |         |    1.800         | us/op |
| TracerBenchmarks.joinWithSpanInScope:p0.95                              | sample |        |     1.700         |        |    1.700         |           |    2.300         |         |    2.300         | us/op |
| TracerBenchmarks.joinWithSpanInScope:p0.99                              | sample |        |     3.600         |        |    3.900         |           |    6.696         |         |    6.696         | us/op |
| TracerBenchmarks.joinWithSpanInScope:p0.999                             | sample |        |    26.880         |        |   36.588         |           |   50.240         |         |   48.064         | us/op |
| TracerBenchmarks.joinWithSpanInScope:p0.9999                            | sample |        |   102.208         |        |  120.568         |           |  139.895         |         |  157.853         | us/op |
| TracerBenchmarks.joinWithSpanInScope:p1.00                              | sample |        | 10584.064         |        | 3506.176         |           | 5734.400         |         | 7266.304         | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage                            | sample | 865058 |     1.284 ± 0.075 | 789384 |    1.567 ± 0.064 |    878595 |    1.601 ± 0.065 |  789720 |    1.493 ± 0.059 | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage:p0.00                      | sample |        |     0.300         |        |    0.300         |           |    0.300         |         |    0.300         | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage:p0.50                      | sample |        |     0.900         |        |    1.100         |           |    1.100         |         |    1.000         | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage:p0.90                      | sample |        |     1.600         |        |    1.900         |           |    2.000         |         |    1.800         | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage:p0.95                      | sample |        |     1.900         |        |    2.300         |           |    2.500         |         |    2.300         | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage:p0.99                      | sample |        |     4.800         |        |    8.496         |           |    7.096         |         |    6.000         | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage:p0.999                     | sample |        |    32.960         |        |   53.888         |           |   49.664         |         |   45.568         | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage:p0.9999                    | sample |        |   102.525         |        |  154.703         |           |  160.584         |         |  143.886         | us/op |
| TracerBenchmarks.joinWithSpanInScope_baggage:p1.00                      | sample |        | 12173.312         |        | 6086.656         |           | 6520.832         |         | 5988.352         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal                       | sample | 951963 |     1.151 ± 0.073 | 904546 |    1.171 ± 0.039 |    779046 |    1.454 ± 0.060 |  861733 |    1.458 ± 0.068 | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.00                 | sample |        |     0.300         |        |    0.300         |           |    0.300         |         |    0.300         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.50                 | sample |        |     0.800         |        |    0.800         |           |    1.000         |         |    1.000         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.90                 | sample |        |     1.400         |        |    1.400         |           |    1.800         |         |    1.800         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.95                 | sample |        |     1.700         |        |    1.700         |           |    2.100         |         |    2.300         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.99                 | sample |        |     3.200         |        |    3.900         |           |    5.696         |         |    5.000         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.999                | sample |        |    26.880         |        |   37.184         |           |   44.541         |         |   42.176         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal:p0.9999               | sample |        |   102.942         |        |  120.134         |           |  159.281         |         |  145.186         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal:p1.00                 | sample |        | 14450.688         |        | 5726.208         |           | 4407.296         |         | 6340.608         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage               | sample | 889799 |     1.431 ± 0.071 | 796288 |    2.603 ± 0.096 |    705532 |    1.728 ± 0.079 |  807849 |    1.360 ± 0.058 | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.00         | sample |        |     0.200         |        |    0.400         |           |    0.300         |         |    0.300         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.50         | sample |        |     0.900         |        |    1.700         |           |    1.200         |         |    1.000         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.90         | sample |        |     1.800         |        |    3.200         |           |    2.100         |         |    1.600         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.95         | sample |        |     2.300         |        |    3.900         |           |    2.700         |         |    1.900         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.99         | sample |        |     6.696         |        |   21.696         |           |   11.088         |         |    3.800         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.999        | sample |        |    48.000         |        |   93.147         |           |   49.822         |         |   35.264         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p0.9999       | sample |        |   140.032         |        |  324.460         |           |  162.249         |         |  133.230         | us/op |
| TracerBenchmarks.joinWithSpanInScope_sampledLocal_baggage:p1.00         | sample |        |  6144.000         |        | 9240.576         |           | 6701.056         |         | 4423.680         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled                          | sample | 815324 |     0.129 ± 0.021 | 897342 |    0.152 ± 0.006 |    991473 |    0.126 ± 0.015 |  881927 |    0.166 ± 0.028 | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled:p0.00                    | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled:p0.50                    | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled:p0.90                    | sample |        |     0.100         |        |    0.200         |           |    0.100         |         |    0.200         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled:p0.95                    | sample |        |     0.200         |        |    0.200         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled:p0.99                    | sample |        |     0.200         |        |    0.400         |           |    0.200         |         |    0.300         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled:p0.999                   | sample |        |     9.488         |        |   16.192         |           |   11.696         |         |   15.296         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled:p0.9999                  | sample |        |    24.992         |        |   38.848         |           |   26.880         |         |   33.307         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled:p1.00                    | sample |        |  2863.104         |        | 1398.784         |           | 3375.104         |         | 3579.904         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage                  | sample | 729570 |     0.253 ± 0.027 | 876918 |    0.407 ± 0.030 |    861679 |    0.294 ± 0.035 |  962574 |    0.363 ± 0.021 | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.00            | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.50            | sample |        |     0.200         |        |    0.300         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.90            | sample |        |     0.300         |        |    0.500         |           |    0.400         |         |    0.400         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.95            | sample |        |     0.400         |        |    0.600         |           |    0.500         |         |    0.500         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.99            | sample |        |     0.800         |        |    1.100         |           |    0.800         |         |    0.900         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.999           | sample |        |    16.000         |        |   22.696         |           |   18.400         |         |   23.872         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p0.9999          | sample |        |    34.376         |        |   87.975         |           |   44.480         |         |   94.815         | us/op |
| TracerBenchmarks.joinWithSpanInScope_unsampled_baggage:p1.00            | sample |        |  2666.496         |        | 3280.896         |           | 4702.208         |         | 3420.160         | us/op |
| TracerBenchmarks.newChildWithSpanInScope                                | sample | 688420 |     0.826 ± 0.053 | 702916 |    0.916 ± 0.051 |    841655 |    0.947 ± 0.041 |  764203 |    0.935 ± 0.056 | us/op |
| TracerBenchmarks.newChildWithSpanInScope:p0.00                          | sample |        |     0.399         |        |    0.400         |           |    0.400         |         |    0.399         | us/op |
| TracerBenchmarks.newChildWithSpanInScope:p0.50                          | sample |        |     0.600         |        |    0.700         |           |    0.700         |         |    0.700         | us/op |
| TracerBenchmarks.newChildWithSpanInScope:p0.90                          | sample |        |     0.900         |        |    1.000         |           |    1.000         |         |    1.000         | us/op |
| TracerBenchmarks.newChildWithSpanInScope:p0.95                          | sample |        |     1.000         |        |    1.100         |           |    1.200         |         |    1.100         | us/op |
| TracerBenchmarks.newChildWithSpanInScope:p0.99                          | sample |        |     1.300         |        |    1.700         |           |    2.400         |         |    1.600         | us/op |
| TracerBenchmarks.newChildWithSpanInScope:p0.999                         | sample |        |    19.488         |        |   23.680         |           |   24.576         |         |   22.976         | us/op |
| TracerBenchmarks.newChildWithSpanInScope:p0.9999                        | sample |        |    68.386         |        |   77.339         |           |   88.002         |         |   92.019         | us/op |
| TracerBenchmarks.newChildWithSpanInScope:p1.00                          | sample |        |  3301.376         |        | 3284.992         |           | 3567.616         |         | 4382.720         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage                        | sample | 843103 |     0.938 ± 0.042 | 794588 |    1.573 ± 0.073 |    980728 |    1.078 ± 0.040 |  961350 |    1.150 ± 0.051 | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage:p0.00                  | sample |        |     0.400         |        |    0.499         |           |    0.400         |         |    0.400         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage:p0.50                  | sample |        |     0.800         |        |    1.100         |           |    0.900         |         |    0.900         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage:p0.90                  | sample |        |     1.000         |        |    1.800         |           |    1.200         |         |    1.300         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage:p0.95                  | sample |        |     1.100         |        |    2.100         |           |    1.400         |         |    1.500         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage:p0.99                  | sample |        |     1.500         |        |    5.000         |           |    2.300         |         |    2.400         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage:p0.999                 | sample |        |    19.200         |        |   50.240         |           |   26.496         |         |   33.472         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage:p0.9999                | sample |        |    57.132         |        |  236.565         |           |   84.661         |         |  124.015         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_baggage:p1.00                  | sample |        |  2596.864         |        | 4603.904         |           | 4669.440         |         | 6135.808         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal                   | sample | 672946 |     0.828 ± 0.049 | 783725 |    0.945 ± 0.042 |    833441 |    0.996 ± 0.041 |  967233 |    1.060 ± 0.053 | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.00             | sample |        |     0.400         |        |    0.400         |           |    0.400         |         |    0.400         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.50             | sample |        |     0.600         |        |    0.700         |           |    0.800         |         |    0.800         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.90             | sample |        |     0.900         |        |    1.100         |           |    1.100         |         |    1.200         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.95             | sample |        |     1.000         |        |    1.200         |           |    1.300         |         |    1.400         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.99             | sample |        |     1.400         |        |    1.900         |           |    2.200         |         |    2.200         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.999            | sample |        |    19.296         |        |   25.184         |           |   26.272         |         |   27.680         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p0.9999           | sample |        |    51.068         |        |   75.216         |           |   78.164         |         |   89.804         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal:p1.00             | sample |        |  3178.496         |        | 2465.792         |           | 4104.192         |         | 5693.440         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage           | sample | 934521 |     0.926 ± 0.037 | 830123 |    1.368 ± 0.072 |    002776 |    1.023 ± 0.040 | 1005813 |    1.070 ± 0.046 | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.00     | sample |        |     0.400         |        |    0.400         |           |    0.400         |         |    0.400         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.50     | sample |        |     0.800         |        |    1.000         |           |    0.800         |         |    0.800         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.90     | sample |        |     1.100         |        |    1.500         |           |    1.100         |         |    1.200         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.95     | sample |        |     1.200         |        |    1.800         |           |    1.300         |         |    1.400         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.99     | sample |        |     1.300         |        |    3.500         |           |    1.900         |         |    2.000         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.999    | sample |        |    18.688         |        |   40.768         |           |   23.584         |         |   25.600         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p0.9999   | sample |        |    54.924         |        |  132.096         |           |   69.917         |         |   84.331         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_sampledLocal_baggage:p1.00     | sample |        |  3756.032         |        | 7651.328         |           | 4947.968         |         | 3813.376         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled                      | sample | 953007 |     0.083 ± 0.018 | 897578 |    0.109 ± 0.015 |    834073 |    0.130 ± 0.011 |  877147 |    0.143 ± 0.013 | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.00                | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.50                | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.90                | sample |        |     0.100         |        |    0.100         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.95                | sample |        |     0.100         |        |    0.100         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.99                | sample |        |     0.100         |        |    0.300         |           |    0.300         |         |    0.300         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.999               | sample |        |     1.099         |        |   11.888         |           |   12.800         |         |   13.296         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled:p0.9999              | sample |        |    19.072         |        |   37.502         |           |   27.948         |         |   38.080         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled:p1.00                | sample |        |  2867.200         |        | 3952.640         |           | 2592.768         |         | 3276.800         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage              | sample | 834183 |     0.247 ± 0.026 | 810389 |    0.550 ± 0.055 |    821449 |    0.227 ± 0.022 |  848617 |    0.472 ± 0.028 | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.00        | sample |        |       ≈ 0         |        |    0.100         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.50        | sample |        |     0.200         |        |    0.300         |           |    0.200         |         |    0.300         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.90        | sample |        |     0.300         |        |    0.600         |           |    0.200         |         |    0.600         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.95        | sample |        |     0.300         |        |    0.700         |           |    0.300         |         |    0.700         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.99        | sample |        |     0.400         |        |    1.100         |           |    0.400         |         |    1.100         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.999       | sample |        |    15.678         |        |   31.680         |           |   16.288         |         |   26.976         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p0.9999      | sample |        |    36.618         |        |  126.459         |           |   35.831         |         |  103.442         | us/op |
| TracerBenchmarks.newChildWithSpanInScope_unsampled_baggage:p1.00        | sample |        |  3059.712         |        | 7184.384         |           | 3227.648         |         | 4333.568         | us/op |
| TracerBenchmarks.nextWithSpanInScope                                    | sample | 699486 |     0.782 ± 0.043 | 870070 |    1.012 ± 0.062 |    893696 |    0.991 ± 0.052 | 1000402 |    1.075 ± 0.041 | us/op |
| TracerBenchmarks.nextWithSpanInScope:p0.00                              | sample |        |     0.400         |        |    0.400         |           |    0.400         |         |    0.400         | us/op |
| TracerBenchmarks.nextWithSpanInScope:p0.50                              | sample |        |     0.600         |        |    0.700         |           |    0.800         |         |    0.800         | us/op |
| TracerBenchmarks.nextWithSpanInScope:p0.90                              | sample |        |     0.800         |        |    1.100         |           |    1.100         |         |    1.300         | us/op |
| TracerBenchmarks.nextWithSpanInScope:p0.95                              | sample |        |     1.000         |        |    1.300         |           |    1.200         |         |    1.500         | us/op |
| TracerBenchmarks.nextWithSpanInScope:p0.99                              | sample |        |     1.200         |        |    2.000         |           |    1.900         |         |    2.200         | us/op |
| TracerBenchmarks.nextWithSpanInScope:p0.999                             | sample |        |    18.496         |        |   30.272         |           |   24.288         |         |   30.688         | us/op |
| TracerBenchmarks.nextWithSpanInScope:p0.9999                            | sample |        |    54.625         |        |  101.631         |           |   60.601         |         |  112.855         | us/op |
| TracerBenchmarks.nextWithSpanInScope:p1.00                              | sample |        |  3108.864         |        | 5464.064         |           | 3878.912         |         | 3780.608         | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage                            | sample | 912570 |     0.923 ± 0.037 | 833799 |    1.365 ± 0.064 |    967209 |    1.078 ± 0.044 |  829780 |    1.551 ± 0.077 | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage:p0.00                      | sample |        |     0.400         |        |    0.500         |           |    0.400         |         |    0.400         | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage:p0.50                      | sample |        |     0.800         |        |    1.000         |           |    0.800         |         |    1.100         | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage:p0.90                      | sample |        |     1.100         |        |    1.500         |           |    1.200         |         |    1.800         | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage:p0.95                      | sample |        |     1.100         |        |    1.800         |           |    1.400         |         |    2.100         | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage:p0.99                      | sample |        |     1.500         |        |    3.200         |           |    2.200         |         |    4.200         | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage:p0.999                     | sample |        |    18.688         |        |   40.768         |           |   26.373         |         |   45.952         | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage:p0.9999                    | sample |        |    51.085         |        |  153.564         |           |   81.494         |         |  178.182         | us/op |
| TracerBenchmarks.nextWithSpanInScope_baggage:p1.00                      | sample |        |  3600.384         |        | 5144.576         |           | 4001.792         |         | 6684.672         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled                          | sample | 971978 |     0.108 ± 0.002 | 776152 |    0.127 ± 0.005 |    953758 |    0.129 ± 0.003 |  849834 |    0.186 ± 0.015 | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled:p0.00                    | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled:p0.50                    | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled:p0.90                    | sample |        |     0.200         |        |    0.200         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled:p0.95                    | sample |        |     0.200         |        |    0.200         |           |    0.200         |         |    0.300         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled:p0.99                    | sample |        |     0.200         |        |    0.300         |           |    0.300         |         |    0.400         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled:p0.999                   | sample |        |     3.800         |        |   15.200         |           |   11.600         |         |   19.872         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled:p0.9999                  | sample |        |    22.496         |        |   41.984         |           |   27.512         |         |   63.680         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled:p1.00                    | sample |        |   155.648         |        |  449.024         |           |  308.736         |         | 2662.400         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage                  | sample | 807001 |     0.282 ± 0.043 | 964087 |    0.412 ± 0.044 |    872671 |    0.309 ± 0.034 |  949183 |    0.402 ± 0.052 | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.00            | sample |        |     0.099         |        |    0.100         |           |    0.099         |         |    0.100         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.50            | sample |        |     0.200         |        |    0.200         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.90            | sample |        |     0.300         |        |    0.500         |           |    0.300         |         |    0.400         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.95            | sample |        |     0.300         |        |    0.600         |           |    0.400         |         |    0.500         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.99            | sample |        |     0.400         |        |    1.000         |           |    0.600         |         |    0.700         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.999           | sample |        |    12.992         |        |   24.192         |           |   18.272         |         |   22.080         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p0.9999          | sample |        |    32.538         |        |   84.399         |           |   45.005         |         |   63.082         | us/op |
| TracerBenchmarks.nextWithSpanInScope_unsampled_baggage:p1.00            | sample |        |  3428.352         |        | 5873.664         |           | 3846.144         |         | 5423.104         | us/op |
| TracerBenchmarks.startScopedSpanWithParent                              | sample | 806921 |     0.696 ± 0.046 | 838624 |    1.380 ± 0.075 |    738942 |    0.863 ± 0.073 |  914953 |    1.041 ± 0.061 | us/op |
| TracerBenchmarks.startScopedSpanWithParent:p0.00                        | sample |        |     0.300         |        |    0.300         |           |    0.300         |         |    0.300         | us/op |
| TracerBenchmarks.startScopedSpanWithParent:p0.50                        | sample |        |     0.500         |        |    0.900         |           |    0.600         |         |    0.700         | us/op |
| TracerBenchmarks.startScopedSpanWithParent:p0.90                        | sample |        |     0.700         |        |    1.700         |           |    0.900         |         |    1.200         | us/op |
| TracerBenchmarks.startScopedSpanWithParent:p0.95                        | sample |        |     0.800         |        |    2.000         |           |    1.000         |         |    1.300         | us/op |
| TracerBenchmarks.startScopedSpanWithParent:p0.99                        | sample |        |     1.000         |        |    3.600         |           |    1.600         |         |    2.100         | us/op |
| TracerBenchmarks.startScopedSpanWithParent:p0.999                       | sample |        |    17.991         |        |   48.960         |           |   23.296         |         |   31.584         | us/op |
| TracerBenchmarks.startScopedSpanWithParent:p0.9999                      | sample |        |    41.339         |        |  161.280         |           |   80.500         |         |  110.786         | us/op |
| TracerBenchmarks.startScopedSpanWithParent:p1.00                        | sample |        |  3317.760         |        | 7872.512         |           | 5251.072         |         | 8118.272         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage                      | sample | 692882 |     0.832 ± 0.056 | 968190 |    1.005 ± 0.052 |    884959 |    1.008 ± 0.047 |  880364 |    1.293 ± 0.066 | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage:p0.00                | sample |        |     0.300         |        |    0.400         |           |    0.300         |         |    0.400         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage:p0.50                | sample |        |     0.600         |        |    0.800         |           |    0.800         |         |    0.900         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage:p0.90                | sample |        |     0.800         |        |    1.100         |           |    1.100         |         |    1.500         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage:p0.95                | sample |        |     0.900         |        |    1.300         |           |    1.300         |         |    1.700         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage:p0.99                | sample |        |     1.200         |        |    1.900         |           |    2.000         |         |    3.000         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage:p0.999               | sample |        |    19.776         |        |   26.592         |           |   27.872         |         |   40.337         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage:p0.9999              | sample |        |    76.854         |        |   96.430         |           |   74.624         |         |  134.647         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_baggage:p1.00                | sample |        |  2965.504         |        | 5013.504         |           | 4505.600         |         | 5939.200         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal                 | sample | 775959 |     0.735 ± 0.046 | 871479 |    1.302 ± 0.089 |    719161 |    0.884 ± 0.066 |  826370 |    0.989 ± 0.045 | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.00           | sample |        |     0.300         |        |    0.300         |           |    0.300         |         |    0.300         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.50           | sample |        |     0.600         |        |    0.900         |           |    0.600         |         |    0.700         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.90           | sample |        |     0.800         |        |    1.500         |           |    0.900         |         |    1.200         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.95           | sample |        |     0.900         |        |    1.800         |           |    1.100         |         |    1.400         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.99           | sample |        |     1.100         |        |    3.000         |           |    1.800         |         |    2.200         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.999          | sample |        |    18.080         |        |   40.768         |           |   25.376         |         |   31.296         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p0.9999         | sample |        |    66.170         |        |  124.397         |           |   72.960         |         |   81.221         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal:p1.00           | sample |        |  2846.720         |        | 8388.608         |           | 5791.744         |         | 4014.080         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage         | sample | 663385 |     0.832 ± 0.043 | 872578 |    1.160 ± 0.059 |    762006 |    0.983 ± 0.059 |  850743 |    1.339 ± 0.072 | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.00   | sample |        |     0.300         |        |    0.400         |           |    0.300         |         |    0.400         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.50   | sample |        |     0.700         |        |    0.800         |           |    0.700         |         |    1.000         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.90   | sample |        |     0.900         |        |    1.300         |           |    1.100         |         |    1.500         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.95   | sample |        |     1.000         |        |    1.600         |           |    1.200         |         |    1.700         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.99   | sample |        |     1.400         |        |    2.700         |           |    2.000         |         |    3.000         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.999  | sample |        |    22.400         |        |   36.544         |           |   25.472         |         |   38.784         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p0.9999 | sample |        |    74.577         |        |  155.772         |           |   77.491         |         |  162.560         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_sampledLocal_baggage:p1.00   | sample |        |  3436.544         |        | 4562.944         |           | 5447.680         |         | 6897.664         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled                    | sample | 915326 |     0.111 ± 0.012 | 873649 |    0.167 ± 0.024 |    940305 |    0.165 ± 0.027 |  738457 |    0.160 ± 0.031 | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.00              | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.50              | sample |        |     0.100         |        |    0.100         |           |    0.100         |         |    0.100         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.90              | sample |        |     0.100         |        |    0.200         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.95              | sample |        |     0.200         |        |    0.200         |           |    0.200         |         |    0.200         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.99              | sample |        |     0.200         |        |    0.300         |           |    0.300         |         |    0.300         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.999             | sample |        |     7.265         |        |   19.234         |           |   15.966         |         |   17.184         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled:p0.9999            | sample |        |    22.880         |        |   53.584         |           |   35.444         |         |   44.810         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled:p1.00              | sample |        |  2347.008         |        | 4194.304         |           | 4431.872         |         | 5881.856         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage            | sample | 814910 |     0.247 ± 0.035 | 961982 |    0.405 ± 0.037 |    861444 |    0.243 ± 0.022 |  923887 |    0.395 ± 0.018 | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.00      | sample |        |       ≈ 0         |        |      ≈ 0         |           |      ≈ 0         |         |      ≈ 0         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.50      | sample |        |     0.200         |        |    0.300         |           |    0.200         |         |    0.300         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.90      | sample |        |     0.300         |        |    0.400         |           |    0.300         |         |    0.500         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.95      | sample |        |     0.300         |        |    0.500         |           |    0.300         |         |    0.600         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.99      | sample |        |     0.400         |        |    0.800         |           |    0.400         |         |    0.900         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.999     | sample |        |    14.592         |        |   21.888         |           |   16.480         |         |   22.091         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p0.9999    | sample |        |    31.749         |        |   77.645         |           |   37.870         |         |   96.896         | us/op |
| TracerBenchmarks.startScopedSpanWithParent_unsampled_baggage:p1.00      | sample |        |  3772.416         |        | 5824.512         |           | 2916.352         |         | 2555.904         | us/op |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
codefromthecrypt commented 8 months ago

fyi I'm going to unsubscribe from this topic as I feel like I don't want to watch us overcompensate for a "problem" put out of context. This is not about you, rather my frustration of being the recipient of work on something I don't even use, despite the number of paid employees of tracing that exist.

Here's the summary of suggestions:

  1. measure actual micrometer and see how much overhead is made https://github.com/openzipkin/brave-example/tree/master/webflux6-micrometer
  2. consider what's going on... span finish/abandon when if using zipkin asyncreporter the span is sized for marshalled to avoid OOM on backlog. Only here is there a time and I bet it isn't the lions share of overhead.
  3. consider if sampling is used at all, and what would be the effective surface area of this problem if traces were rate limited. Note that current micrometer/spring boot solution no longer includes http sampling which we had before and would reduce this sort of impact especially for low to zero value traces
  4. considering all of that, what is the most surgical way to move forward without adding complexity.
  5. measure the change vs base case in 1 and see if I'm wrong about this being a dominating factor ;)

Maybe, we can change the AsyncReporter to handle this optionally https://github.com/openzipkin/zipkin-reporter-java/blob/master/brave/src/main/java/zipkin2/reporter/brave/AsyncZipkinSpanHandler.java#L212

Here, instead of doing "report" inline (which sizes the span), we can make a copy and return false (meaning this reporter must be the end of the line, which it already should be.

After the copy is done (still in the lock), put it into a thread (maybe the existing reporter thread) to size or drop as needed. There are benchmarks in that project, so you can see how long the lock is being held for. Of course if a tracer adds a ton of tags it may be a lot longer with the current approach, but not as long as actually serializing or sending the span.

EOF and good luck!