search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
573 stars 284 forks source link

WIP attempt to support metadata in newer spring-kafka #7565

Open roisinlh opened 3 weeks ago

roisinlh commented 3 weeks ago

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

pr-commenter[bot] commented 3 weeks ago

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master roisin.luddyhughes/spring-kafka-use-group-metadata
git_commit_date 1725455874 1725464361
git_commit_sha 86d01e6f144032b102a9c4dfca80ee491532fd0a 564a1c0b3f80e63783d17a38bc1a94788e1e4654
See matching parameters | |Baseline |Candidate | |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |ci_job_date |1725465389 |1725465389 | |ci_job_id |627452599 |627452599 | |ci_pipeline_id|43528899 |43528899 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | |jdkVersion |11.0.21 |11.0.21 | |jmhVersion |1.36 |1.36 | |jvm |/usr/lib/jvm/java-11-openjdk-amd64/bin/java |/usr/lib/jvm/java-11-openjdk-amd64/bin/java | |jvmArgs |-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant|-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant| |vmName |OpenJDK 64-Bit Server VM |OpenJDK 64-Bit Server VM | |vmVersion |11.0.21+9-post-Ubuntu-0ubuntu122.04 |11.0.21+9-post-Ubuntu-0ubuntu122.04 |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results |scenario |Δ mean throughput | |--------------------------------------------------------------------------------|-----------------------------------------------------------------------| |scenario:not-instrumented/KafkaProduceBenchmark.benchProduce |unsure
[-70806.657op/s; -14743.988op/s] or [-3.859%; -0.803%]| |scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce|same | |scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce |same |
pr-commenter[bot] commented 3 weeks ago

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master roisin.luddyhughes/spring-kafka-use-group-metadata
git_commit_date 1725455874 1725464361
git_commit_sha 86d01e6f14 564a1c0b3f
release_version 1.40.0-SNAPSHOT~86d01e6f14 1.40.0-SNAPSHOT~564a1c0b3f
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1725466740 |1725466740 | |ci_job_id |627452591 |627452591 | |ci_pipeline_id|43528899 |43528899 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |module |Agent |Agent | |parent |None |None | |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for insecure-bank ```mermaid gantt title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~564a1c0b3f, baseline=1.40.0-SNAPSHOT~86d01e6f14 dateFormat X axisFormat %s section tracing Agent [baseline] (1.051 s) : 0, 1051456 Total [baseline] (8.493 s) : 0, 8493492 Agent [candidate] (1.064 s) : 0, 1063966 Total [candidate] (8.507 s) : 0, 8506503 section iast Agent [baseline] (1.183 s) : 0, 1182834 Total [baseline] (9.027 s) : 0, 9026918 Agent [candidate] (1.176 s) : 0, 1175665 Total [candidate] (8.967 s) : 0, 8966628 section iast_HARDCODED_SECRET_DISABLED Agent [baseline] (1.175 s) : 0, 1175324 Total [baseline] (8.939 s) : 0, 8938750 Agent [candidate] (1.176 s) : 0, 1176387 Total [candidate] (8.942 s) : 0, 8941779 section iast_TELEMETRY_OFF Agent [baseline] (1.172 s) : 0, 1172397 Total [baseline] (9.029 s) : 0, 9028597 Agent [candidate] (1.181 s) : 0, 1181162 Total [candidate] (8.957 s) : 0, 8956778 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.051 s |-| |Agent|iast|1.183 s |131.378 ms (12.5%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.175 s |123.868 ms (11.8%)| |Agent|iast_TELEMETRY_OFF|1.172 s |120.941 ms (11.5%)| |Total|tracing|8.493 s |-| |Total|iast|9.027 s |533.426 ms (6.3%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.939 s |445.258 ms (5.2%)| |Total|iast_TELEMETRY_OFF|9.029 s |535.104 ms (6.3%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.064 s |-| |Agent|iast|1.176 s |111.699 ms (10.5%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.176 s |112.421 ms (10.6%)| |Agent|iast_TELEMETRY_OFF|1.181 s |117.196 ms (11.0%)| |Total|tracing|8.507 s |-| |Total|iast|8.967 s |460.125 ms (5.4%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.942 s |435.276 ms (5.1%)| |Total|iast_TELEMETRY_OFF|8.957 s |450.275 ms (5.3%)| ```mermaid gantt title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~564a1c0b3f, baseline=1.40.0-SNAPSHOT~86d01e6f14 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (670.616 ms) : 0, 670616 BytebuddyAgent [candidate] (679.22 ms) : 0, 679220 GlobalTracer [baseline] (307.634 ms) : 0, 307634 GlobalTracer [candidate] (311.126 ms) : 0, 311126 AppSec [baseline] (51.433 ms) : 0, 51433 AppSec [candidate] (51.636 ms) : 0, 51636 Remote Config [baseline] (687.195 µs) : 0, 687 Remote Config [candidate] (673.889 µs) : 0, 674 Telemetry [baseline] (7.489 ms) : 0, 7489 Telemetry [candidate] (7.493 ms) : 0, 7493 section iast BytebuddyAgent [baseline] (787.365 ms) : 0, 787365 BytebuddyAgent [candidate] (782.488 ms) : 0, 782488 GlobalTracer [baseline] (298.369 ms) : 0, 298369 GlobalTracer [candidate] (296.678 ms) : 0, 296678 AppSec [baseline] (52.063 ms) : 0, 52063 AppSec [candidate] (52.122 ms) : 0, 52122 Remote Config [baseline] (581.4 µs) : 0, 581 Remote Config [candidate] (596.357 µs) : 0, 596 Telemetry [baseline] (8.044 ms) : 0, 8044 Telemetry [candidate] (8.112 ms) : 0, 8112 IAST [baseline] (22.709 ms) : 0, 22709 IAST [candidate] (22.011 ms) : 0, 22011 section iast_HARDCODED_SECRET_DISABLED BytebuddyAgent [baseline] (781.837 ms) : 0, 781837 BytebuddyAgent [candidate] (781.609 ms) : 0, 781609 GlobalTracer [baseline] (296.352 ms) : 0, 296352 GlobalTracer [candidate] (297.125 ms) : 0, 297125 AppSec [baseline] (52.908 ms) : 0, 52908 AppSec [candidate] (51.871 ms) : 0, 51871 Remote Config [baseline] (577.325 µs) : 0, 577 Remote Config [candidate] (589.188 µs) : 0, 589 Telemetry [baseline] (8.089 ms) : 0, 8089 Telemetry [candidate] (7.296 ms) : 0, 7296 IAST [baseline] (21.94 ms) : 0, 21940 IAST [candidate] (24.222 ms) : 0, 24222 section iast_TELEMETRY_OFF BytebuddyAgent [baseline] (778.726 ms) : 0, 778726 BytebuddyAgent [candidate] (785.134 ms) : 0, 785134 GlobalTracer [baseline] (296.465 ms) : 0, 296465 GlobalTracer [candidate] (298.802 ms) : 0, 298802 AppSec [baseline] (50.693 ms) : 0, 50693 AppSec [candidate] (51.836 ms) : 0, 51836 Remote Config [baseline] (584.715 µs) : 0, 585 Remote Config [candidate] (579.498 µs) : 0, 579 Telemetry [baseline] (7.346 ms) : 0, 7346 Telemetry [candidate] (7.206 ms) : 0, 7206 IAST [baseline] (24.954 ms) : 0, 24954 IAST [candidate] (23.797 ms) : 0, 23797 ```
Startup time reports for petclinic ```mermaid gantt title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~564a1c0b3f, baseline=1.40.0-SNAPSHOT~86d01e6f14 dateFormat X axisFormat %s section tracing Agent [baseline] (1.059 s) : 0, 1058877 Total [baseline] (10.358 s) : 0, 10357673 Agent [candidate] (1.052 s) : 0, 1051807 Total [candidate] (10.347 s) : 0, 10347239 section appsec Agent [baseline] (1.186 s) : 0, 1186177 Total [baseline] (10.568 s) : 0, 10567766 Agent [candidate] (1.181 s) : 0, 1181087 Total [candidate] (10.582 s) : 0, 10581636 section iast Agent [baseline] (1.177 s) : 0, 1177389 Total [baseline] (10.854 s) : 0, 10853712 Agent [candidate] (1.176 s) : 0, 1175762 Total [candidate] (10.809 s) : 0, 10809153 section profiling Agent [baseline] (1.257 s) : 0, 1256812 Total [baseline] (10.634 s) : 0, 10634122 Agent [candidate] (1.25 s) : 0, 1249557 Total [candidate] (10.64 s) : 0, 10640159 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.059 s |-| |Agent|appsec|1.186 s |127.3 ms (12.0%)| |Agent|iast|1.177 s |118.512 ms (11.2%)| |Agent|profiling|1.257 s |197.934 ms (18.7%)| |Total|tracing|10.358 s |-| |Total|appsec|10.568 s |210.092 ms (2.0%)| |Total|iast|10.854 s |496.039 ms (4.8%)| |Total|profiling|10.634 s |276.449 ms (2.7%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.052 s |-| |Agent|appsec|1.181 s |129.28 ms (12.3%)| |Agent|iast|1.176 s |123.954 ms (11.8%)| |Agent|profiling|1.25 s |197.75 ms (18.8%)| |Total|tracing|10.347 s |-| |Total|appsec|10.582 s |234.398 ms (2.3%)| |Total|iast|10.809 s |461.915 ms (4.5%)| |Total|profiling|10.64 s |292.921 ms (2.8%)| ```mermaid gantt title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~564a1c0b3f, baseline=1.40.0-SNAPSHOT~86d01e6f14 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (675.723 ms) : 0, 675723 BytebuddyAgent [candidate] (670.78 ms) : 0, 670780 GlobalTracer [baseline] (309.284 ms) : 0, 309284 GlobalTracer [candidate] (307.902 ms) : 0, 307902 AppSec [baseline] (51.91 ms) : 0, 51910 AppSec [candidate] (51.383 ms) : 0, 51383 Remote Config [baseline] (693.159 µs) : 0, 693 Remote Config [candidate] (670.036 µs) : 0, 670 Telemetry [baseline] (7.607 ms) : 0, 7607 Telemetry [candidate] (7.447 ms) : 0, 7447 section appsec BytebuddyAgent [baseline] (692.352 ms) : 0, 692352 BytebuddyAgent [candidate] (688.903 ms) : 0, 688903 GlobalTracer [baseline] (301.451 ms) : 0, 301451 GlobalTracer [candidate] (299.951 ms) : 0, 299951 AppSec [baseline] (158.99 ms) : 0, 158990 AppSec [candidate] (159.006 ms) : 0, 159006 Remote Config [baseline] (622.385 µs) : 0, 622 Remote Config [candidate] (618.904 µs) : 0, 619 Telemetry [baseline] (8.895 ms) : 0, 8895 Telemetry [candidate] (9.233 ms) : 0, 9233 IAST [baseline] (20.514 ms) : 0, 20514 IAST [candidate] (19.671 ms) : 0, 19671 section iast BytebuddyAgent [baseline] (782.881 ms) : 0, 782881 BytebuddyAgent [candidate] (780.612 ms) : 0, 780612 GlobalTracer [baseline] (296.937 ms) : 0, 296937 GlobalTracer [candidate] (297.981 ms) : 0, 297981 AppSec [baseline] (51.382 ms) : 0, 51382 AppSec [candidate] (54.404 ms) : 0, 54404 Remote Config [baseline] (615.219 µs) : 0, 615 Remote Config [candidate] (592.251 µs) : 0, 592 Telemetry [baseline] (7.383 ms) : 0, 7383 Telemetry [candidate] (7.37 ms) : 0, 7370 IAST [baseline] (24.584 ms) : 0, 24584 IAST [candidate] (21.199 ms) : 0, 21199 section profiling ProfilingAgent [baseline] (96.645 ms) : 0, 96645 ProfilingAgent [candidate] (95.239 ms) : 0, 95239 BytebuddyAgent [baseline] (669.934 ms) : 0, 669934 BytebuddyAgent [candidate] (666.302 ms) : 0, 666302 GlobalTracer [baseline] (391.613 ms) : 0, 391613 GlobalTracer [candidate] (389.978 ms) : 0, 389978 AppSec [baseline] (52.395 ms) : 0, 52395 AppSec [candidate] (52.029 ms) : 0, 52029 Remote Config [baseline] (684.617 µs) : 0, 685 Remote Config [candidate] (684.057 µs) : 0, 684 Telemetry [baseline] (7.447 ms) : 0, 7447 Telemetry [candidate] (7.398 ms) : 0, 7398 Profiling [baseline] (96.669 ms) : 0, 96669 Profiling [candidate] (95.262 ms) : 0, 95262 ```

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-04T15:49:42 2024-09-04T15:56:30
git_branch master roisin.luddyhughes/spring-kafka-use-group-metadata
git_commit_date 1725455874 1725464361
git_commit_sha 86d01e6f14 564a1c0b3f
release_version 1.40.0-SNAPSHOT~86d01e6f14 1.40.0-SNAPSHOT~564a1c0b3f
start_time 2024-09-04T15:49:29 2024-09-04T15:56:17
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1725465734 |1725465734 | |ci_job_id |627452593 |627452593 | |ci_pipeline_id|43528899 |43528899 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 19 unstable metrics.

Request duration reports for insecure-bank ```mermaid gantt title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~564a1c0b3f, baseline=1.40.0-SNAPSHOT~86d01e6f14 dateFormat X axisFormat %s section baseline no_agent (362.785 µs) : 343, 382 . : milestone, 363, iast (479.587 µs) : 458, 501 . : milestone, 480, iast_FULL (546.365 µs) : 525, 568 . : milestone, 546, iast_GLOBAL (505.232 µs) : 484, 527 . : milestone, 505, iast_HARDCODED_SECRET_DISABLED (477.717 µs) : 456, 500 . : milestone, 478, iast_INACTIVE (440.208 µs) : 419, 461 . : milestone, 440, iast_TELEMETRY_OFF (469.045 µs) : 446, 492 . : milestone, 469, tracing (432.229 µs) : 412, 453 . : milestone, 432, section candidate no_agent (364.879 µs) : 345, 385 . : milestone, 365, iast (478.786 µs) : 456, 501 . : milestone, 479, iast_FULL (546.944 µs) : 526, 568 . : milestone, 547, iast_GLOBAL (491.656 µs) : 471, 513 . : milestone, 492, iast_HARDCODED_SECRET_DISABLED (491.909 µs) : 469, 515 . : milestone, 492, iast_INACTIVE (437.065 µs) : 417, 457 . : milestone, 437, iast_TELEMETRY_OFF (466.499 µs) : 444, 489 . : milestone, 466, tracing (434.162 µs) : 414, 455 . : milestone, 434, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|362.785 µs [343.479 µs, 382.09 µs]|-| |iast|479.587 µs [457.8 µs, 501.373 µs]|116.802 µs (32.2%)| |iast_FULL|546.365 µs [525.183 µs, 567.548 µs]|183.58 µs (50.6%)| |iast_GLOBAL|505.232 µs [483.628 µs, 526.837 µs]|142.447 µs (39.3%)| |iast_HARDCODED_SECRET_DISABLED|477.717 µs [455.724 µs, 499.71 µs]|114.932 µs (31.7%)| |iast_INACTIVE|440.208 µs [419.031 µs, 461.385 µs]|77.423 µs (21.3%)| |iast_TELEMETRY_OFF|469.045 µs [446.27 µs, 491.82 µs]|106.26 µs (29.3%)| |tracing|432.229 µs [411.904 µs, 452.555 µs]|69.445 µs (19.1%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|364.879 µs [344.865 µs, 384.892 µs]|-| |iast|478.786 µs [456.267 µs, 501.305 µs]|113.907 µs (31.2%)| |iast_FULL|546.944 µs [525.948 µs, 567.939 µs]|182.065 µs (49.9%)| |iast_GLOBAL|491.656 µs [470.654 µs, 512.658 µs]|126.778 µs (34.7%)| |iast_HARDCODED_SECRET_DISABLED|491.909 µs [468.874 µs, 514.944 µs]|127.03 µs (34.8%)| |iast_INACTIVE|437.065 µs [416.701 µs, 457.428 µs]|72.186 µs (19.8%)| |iast_TELEMETRY_OFF|466.499 µs [444.36 µs, 488.638 µs]|101.62 µs (27.9%)| |tracing|434.162 µs [413.776 µs, 454.548 µs]|69.284 µs (19.0%)|
Request duration reports for petclinic ```mermaid gantt title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~564a1c0b3f, baseline=1.40.0-SNAPSHOT~86d01e6f14 dateFormat X axisFormat %s section baseline no_agent (1.346 ms) : 1327, 1365 . : milestone, 1346, appsec (1.726 ms) : 1701, 1750 . : milestone, 1726, appsec_no_iast (1.728 ms) : 1704, 1752 . : milestone, 1728, iast (1.454 ms) : 1431, 1476 . : milestone, 1454, profiling (1.523 ms) : 1499, 1548 . : milestone, 1523, tracing (1.474 ms) : 1450, 1498 . : milestone, 1474, section candidate no_agent (1.343 ms) : 1324, 1363 . : milestone, 1343, appsec (1.731 ms) : 1707, 1755 . : milestone, 1731, appsec_no_iast (1.731 ms) : 1708, 1755 . : milestone, 1731, iast (1.476 ms) : 1454, 1498 . : milestone, 1476, profiling (1.475 ms) : 1452, 1498 . : milestone, 1475, tracing (1.463 ms) : 1439, 1487 . : milestone, 1463, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.346 ms [1.327 ms, 1.365 ms]|-| |appsec|1.726 ms [1.701 ms, 1.75 ms]|379.802 µs (28.2%)| |appsec_no_iast|1.728 ms [1.704 ms, 1.752 ms]|382.415 µs (28.4%)| |iast|1.454 ms [1.431 ms, 1.476 ms]|107.883 µs (8.0%)| |profiling|1.523 ms [1.499 ms, 1.548 ms]|177.533 µs (13.2%)| |tracing|1.474 ms [1.45 ms, 1.498 ms]|128.463 µs (9.5%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.343 ms [1.324 ms, 1.363 ms]|-| |appsec|1.731 ms [1.707 ms, 1.755 ms]|387.762 µs (28.9%)| |appsec_no_iast|1.731 ms [1.708 ms, 1.755 ms]|388.084 µs (28.9%)| |iast|1.476 ms [1.454 ms, 1.498 ms]|133.169 µs (9.9%)| |profiling|1.475 ms [1.452 ms, 1.498 ms]|131.809 µs (9.8%)| |tracing|1.463 ms [1.439 ms, 1.487 ms]|119.465 µs (8.9%)|

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master roisin.luddyhughes/spring-kafka-use-group-metadata
git_commit_date 1725455874 1725464361
git_commit_sha 86d01e6f14 564a1c0b3f
release_version 1.40.0-SNAPSHOT~86d01e6f14 1.40.0-SNAPSHOT~564a1c0b3f
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |biojava |biojava | |ci_job_date |1725466238 |1725466238 | |ci_job_id |627452596 |627452596 | |ci_pipeline_id|43528899 |43528899 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |appsec |appsec |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava ```mermaid gantt title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~564a1c0b3f, baseline=1.40.0-SNAPSHOT~86d01e6f14 dateFormat X axisFormat %s section baseline no_agent (15.345 s) : 15345000, 15345000 . : milestone, 15345000, appsec (15.044 s) : 15044000, 15044000 . : milestone, 15044000, iast (18.722 s) : 18722000, 18722000 . : milestone, 18722000, iast_GLOBAL (18.02 s) : 18020000, 18020000 . : milestone, 18020000, profiling (15.462 s) : 15462000, 15462000 . : milestone, 15462000, tracing (15.0 s) : 15000000, 15000000 . : milestone, 15000000, section candidate no_agent (15.437 s) : 15437000, 15437000 . : milestone, 15437000, appsec (15.205 s) : 15205000, 15205000 . : milestone, 15205000, iast (18.876 s) : 18876000, 18876000 . : milestone, 18876000, iast_GLOBAL (17.818 s) : 17818000, 17818000 . : milestone, 17818000, profiling (15.438 s) : 15438000, 15438000 . : milestone, 15438000, tracing (14.93 s) : 14930000, 14930000 . : milestone, 14930000, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.345 s [15.345 s, 15.345 s]|-| |appsec|15.044 s [15.044 s, 15.044 s]|-301.0 ms (-2.0%)| |iast|18.722 s [18.722 s, 18.722 s]|3.377 s (22.0%)| |iast_GLOBAL|18.02 s [18.02 s, 18.02 s]|2.675 s (17.4%)| |profiling|15.462 s [15.462 s, 15.462 s]|117.0 ms (0.8%)| |tracing|15.0 s [15.0 s, 15.0 s]|-345.0 ms (-2.2%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.437 s [15.437 s, 15.437 s]|-| |appsec|15.205 s [15.205 s, 15.205 s]|-232.0 ms (-1.5%)| |iast|18.876 s [18.876 s, 18.876 s]|3.439 s (22.3%)| |iast_GLOBAL|17.818 s [17.818 s, 17.818 s]|2.381 s (15.4%)| |profiling|15.438 s [15.438 s, 15.438 s]|1.0 ms (0.0%)| |tracing|14.93 s [14.93 s, 14.93 s]|-507.0 ms (-3.3%)|
Execution time for tomcat ```mermaid gantt title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~564a1c0b3f, baseline=1.40.0-SNAPSHOT~86d01e6f14 dateFormat X axisFormat %s section baseline no_agent (1.455 ms) : 1444, 1467 . : milestone, 1455, appsec (2.231 ms) : 2196, 2267 . : milestone, 2231, iast (1.963 ms) : 1921, 2006 . : milestone, 1963, iast_GLOBAL (2.012 ms) : 1968, 2056 . : milestone, 2012, profiling (1.858 ms) : 1824, 1892 . : milestone, 1858, tracing (1.835 ms) : 1802, 1868 . : milestone, 1835, section candidate no_agent (1.456 ms) : 1444, 1467 . : milestone, 1456, appsec (2.208 ms) : 2174, 2243 . : milestone, 2208, iast (1.965 ms) : 1922, 2008 . : milestone, 1965, iast_GLOBAL (2.026 ms) : 1983, 2070 . : milestone, 2026, profiling (1.856 ms) : 1822, 1890 . : milestone, 1856, tracing (1.836 ms) : 1803, 1869 . : milestone, 1836, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.455 ms [1.444 ms, 1.467 ms]|-| |appsec|2.231 ms [2.196 ms, 2.267 ms]|775.974 µs (53.3%)| |iast|1.963 ms [1.921 ms, 2.006 ms]|507.747 µs (34.9%)| |iast_GLOBAL|2.012 ms [1.968 ms, 2.056 ms]|556.816 µs (38.3%)| |profiling|1.858 ms [1.824 ms, 1.892 ms]|402.921 µs (27.7%)| |tracing|1.835 ms [1.802 ms, 1.868 ms]|379.768 µs (26.1%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.456 ms [1.444 ms, 1.467 ms]|-| |appsec|2.208 ms [2.174 ms, 2.243 ms]|752.509 µs (51.7%)| |iast|1.965 ms [1.922 ms, 2.008 ms]|509.617 µs (35.0%)| |iast_GLOBAL|2.026 ms [1.983 ms, 2.07 ms]|570.734 µs (39.2%)| |profiling|1.856 ms [1.822 ms, 1.89 ms]|399.955 µs (27.5%)| |tracing|1.836 ms [1.803 ms, 1.869 ms]|379.962 µs (26.1%)|
pr-commenter[bot] commented 3 weeks ago

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master roisin.luddyhughes/spring-kafka-use-group-metadata
git_commit_date 1725455874 1725464361
git_commit_sha 86d01e6f144032b102a9c4dfca80ee491532fd0a 564a1c0b3f80e63783d17a38bc1a94788e1e4654
See matching parameters | |Baseline |Candidate | |---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |ci_job_date |1725465429 |1725465429 | |ci_job_id |627452601 |627452601 | |ci_pipeline_id|43528899 |43528899 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | |jdkVersion |11.0.21 |11.0.21 | |jmhVersion |1.36 |1.36 | |jvm |/usr/lib/jvm/java-11-openjdk-amd64/bin/java |/usr/lib/jvm/java-11-openjdk-amd64/bin/java | |jvmArgs |-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant|-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant| |vmName |OpenJDK 64-Bit Server VM |OpenJDK 64-Bit Server VM | |vmVersion |11.0.21+9-post-Ubuntu-0ubuntu122.04 |11.0.21+9-post-Ubuntu-0ubuntu122.04 |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results |scenario |Δ mean throughput| |---------------------------------------------------------------------------------|-----------------| |scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume |same | |scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume|same | |scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume |same |