search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
572 stars 284 forks source link

Fix flaky smoke tests #7628

Closed jpbempel closed 1 week ago

jpbempel commented 1 week ago

What Does This Do

fix duplicate installed status sent add more timeout for control server

Motivation

flaky tests

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

pr-commenter[bot] commented 1 week ago

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1726508707 1726509082
end_time 2024-09-16T17:46:21 2024-09-16T17:52:37
git_branch master jpbempel/fix-flaky-smoke-tests
git_commit_sha a369f7b401 9a236deed6062ac883fca9d406dc1b8ac9231ead
start_time 2024-09-16T17:45:08 2024-09-16T17:51:23
See matching parameters | |Baseline |Candidate | |----------------------|----------------------------------------------|----------------------------------------------| |ci_job_id |641036209 |641036209 | |ci_pipeline_id |44439712 |44439712 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |git_commit_date|1726508099 |1726508099 |

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 5 metrics, 5 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+1.404ms; +1.422ms] or [+13.877%; +14.055%]		 worse
[+1.396ms; +1.471ms] or [+13.570%; +14.299%]		 worse
[+1.372ms; +1.482ms] or [+13.226%; +14.292%]		 worse
[+1.287ms; +1.887ms] or [+11.893%; +17.436%]		 worse
[-12.578op/s; -10.953op/s] or [-13.081%; -11.391%]
See unchanged results |scenario |Δ mean agg_http_req_duration_min |Δ mean agg_http_req_duration_p50 |Δ mean agg_http_req_duration_p75 |Δ mean agg_http_req_duration_p99 |Δ mean throughput| |----------------|---------------------------------------------------------------------|----------------------------------------------------------------|-----------------------------------------------------------------|-----------------------------------------------------------------|-----------------| |scenario:noprobe|unstable
[-17211.504ns; +16401.215ns] or [-6.665%; +6.351%]|unstable
[-29.739µs; +24.086µs] or [-10.030%; +8.124%]|unstable
[-41.664µs; +35.631µs] or [-13.448%; +11.501%]|unstable
[-88.888µs; +97.872µs] or [-14.856%; +16.358%]|same | |scenario:basic |same |same |same |unstable
[-37.724µs; +47.279µs] or [-7.252%; +9.088%] |same |
Request duration reports for reports ```mermaid gantt title reports - request duration [CI 0.99] : candidate=None, baseline=None dateFormat X axisFormat %s section baseline noprobe (296.492 µs) : 267, 326 . : milestone, 296, basic (293.9 µs) : 285, 303 . : milestone, 294, loop (10.287 ms) : 10246, 10328 . : milestone, 10287, section candidate noprobe (293.666 µs) : 272, 315 . : milestone, 294, basic (298.742 µs) : 291, 307 . : milestone, 299, loop (11.721 ms) : 11694, 11747 . : milestone, 11721, ``` * **baseline** results |Scenario|Request median duration [CI 0.99]| |---|---| |noprobe|296.492 µs [267.347 µs, 325.637 µs]| |basic|293.9 µs [284.921 µs, 302.879 µs]| |loop|10.287 ms [10.246 ms, 10.328 ms]| * **candidate** results |Scenario|Request median duration [CI 0.99]| |---|---| |noprobe|293.666 µs [272.005 µs, 315.326 µs]| |basic|298.742 µs [290.932 µs, 306.552 µs]| |loop|11.721 ms [11.694 ms, 11.747 ms]|
pr-commenter[bot] commented 1 week ago

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-smoke-tests
git_commit_date 1726504555 1726508099
git_commit_sha a369f7b401 9a236deed6
release_version 1.40.0-SNAPSHOT~a369f7b401 1.40.0-SNAPSHOT~9a236deed6
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1726510539 |1726510539 | |ci_job_id |641036203 |641036203 | |ci_pipeline_id|44439712 |44439712 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |module |Agent |Agent | |parent |None |None | |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for insecure-bank ```mermaid gantt title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~9a236deed6, baseline=1.40.0-SNAPSHOT~a369f7b401 dateFormat X axisFormat %s section tracing Agent [baseline] (1.073 s) : 0, 1072514 Total [baseline] (8.543 s) : 0, 8542791 Agent [candidate] (1.066 s) : 0, 1065715 Total [candidate] (8.517 s) : 0, 8517468 section iast Agent [baseline] (1.191 s) : 0, 1190977 Total [baseline] (8.994 s) : 0, 8993868 Agent [candidate] (1.187 s) : 0, 1187088 Total [candidate] (9.053 s) : 0, 9052710 section iast_HARDCODED_SECRET_DISABLED Agent [baseline] (1.189 s) : 0, 1189037 Total [baseline] (8.964 s) : 0, 8964055 Agent [candidate] (1.191 s) : 0, 1190683 Total [candidate] (9.007 s) : 0, 9006907 section iast_TELEMETRY_OFF Agent [baseline] (1.188 s) : 0, 1188432 Total [baseline] (8.999 s) : 0, 8998936 Agent [candidate] (1.185 s) : 0, 1184607 Total [candidate] (8.971 s) : 0, 8971031 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.073 s |-| |Agent|iast|1.191 s |118.462 ms (11.0%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.189 s |116.523 ms (10.9%)| |Agent|iast_TELEMETRY_OFF|1.188 s |115.917 ms (10.8%)| |Total|tracing|8.543 s |-| |Total|iast|8.994 s |451.076 ms (5.3%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.964 s |421.264 ms (4.9%)| |Total|iast_TELEMETRY_OFF|8.999 s |456.145 ms (5.3%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.066 s |-| |Agent|iast|1.187 s |121.373 ms (11.4%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.191 s |124.968 ms (11.7%)| |Agent|iast_TELEMETRY_OFF|1.185 s |118.892 ms (11.2%)| |Total|tracing|8.517 s |-| |Total|iast|9.053 s |535.242 ms (6.3%)| |Total|iast_HARDCODED_SECRET_DISABLED|9.007 s |489.438 ms (5.7%)| |Total|iast_TELEMETRY_OFF|8.971 s |453.563 ms (5.3%)| ```mermaid gantt title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~9a236deed6, baseline=1.40.0-SNAPSHOT~a369f7b401 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (685.078 ms) : 0, 685078 BytebuddyAgent [candidate] (679.378 ms) : 0, 679378 GlobalTracer [baseline] (312.905 ms) : 0, 312905 GlobalTracer [candidate] (312.039 ms) : 0, 312039 AppSec [baseline] (52.631 ms) : 0, 52631 AppSec [candidate] (52.548 ms) : 0, 52548 Remote Config [baseline] (672.529 µs) : 0, 673 Remote Config [candidate] (661.818 µs) : 0, 662 Telemetry [baseline] (7.489 ms) : 0, 7489 Telemetry [candidate] (7.443 ms) : 0, 7443 section iast BytebuddyAgent [baseline] (792.955 ms) : 0, 792955 BytebuddyAgent [candidate] (790.193 ms) : 0, 790193 GlobalTracer [baseline] (299.23 ms) : 0, 299230 GlobalTracer [candidate] (298.338 ms) : 0, 298338 AppSec [baseline] (53.576 ms) : 0, 53576 AppSec [candidate] (54.067 ms) : 0, 54067 IAST [baseline] (23.639 ms) : 0, 23639 IAST [candidate] (22.723 ms) : 0, 22723 Remote Config [baseline] (607.009 µs) : 0, 607 Remote Config [candidate] (694.141 µs) : 0, 694 Telemetry [baseline] (7.324 ms) : 0, 7324 Telemetry [candidate] (7.428 ms) : 0, 7428 section iast_HARDCODED_SECRET_DISABLED BytebuddyAgent [baseline] (791.802 ms) : 0, 791802 BytebuddyAgent [candidate] (792.541 ms) : 0, 792541 GlobalTracer [baseline] (298.77 ms) : 0, 298770 GlobalTracer [candidate] (299.199 ms) : 0, 299199 AppSec [baseline] (54.425 ms) : 0, 54425 AppSec [candidate] (52.84 ms) : 0, 52840 IAST [baseline] (22.533 ms) : 0, 22533 IAST [candidate] (24.401 ms) : 0, 24401 Remote Config [baseline] (605.093 µs) : 0, 605 Remote Config [candidate] (639.487 µs) : 0, 639 Telemetry [baseline] (7.286 ms) : 0, 7286 Telemetry [candidate] (7.366 ms) : 0, 7366 section iast_TELEMETRY_OFF BytebuddyAgent [baseline] (790.616 ms) : 0, 790616 BytebuddyAgent [candidate] (787.185 ms) : 0, 787185 GlobalTracer [baseline] (299.764 ms) : 0, 299764 GlobalTracer [candidate] (299.054 ms) : 0, 299054 AppSec [baseline] (55.371 ms) : 0, 55371 AppSec [candidate] (53.277 ms) : 0, 53277 IAST [baseline] (20.499 ms) : 0, 20499 IAST [candidate] (23.732 ms) : 0, 23732 Remote Config [baseline] (593.56 µs) : 0, 594 Remote Config [candidate] (598.121 µs) : 0, 598 Telemetry [baseline] (7.962 ms) : 0, 7962 Telemetry [candidate] (7.143 ms) : 0, 7143 ```
Startup time reports for petclinic ```mermaid gantt title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~9a236deed6, baseline=1.40.0-SNAPSHOT~a369f7b401 dateFormat X axisFormat %s section tracing Agent [baseline] (1.065 s) : 0, 1065077 Total [baseline] (10.446 s) : 0, 10445776 Agent [candidate] (1.072 s) : 0, 1072235 Total [candidate] (10.321 s) : 0, 10321226 section appsec Agent [baseline] (1.202 s) : 0, 1202396 Total [baseline] (10.68 s) : 0, 10679604 Agent [candidate] (1.2 s) : 0, 1200399 Total [candidate] (10.623 s) : 0, 10623398 section iast Agent [baseline] (1.19 s) : 0, 1189508 Total [baseline] (10.812 s) : 0, 10812082 Agent [candidate] (1.208 s) : 0, 1208048 Total [candidate] (10.889 s) : 0, 10889039 section profiling Agent [baseline] (1.262 s) : 0, 1262483 Total [baseline] (10.641 s) : 0, 10641280 Agent [candidate] (1.27 s) : 0, 1270067 Total [candidate] (10.663 s) : 0, 10663438 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.065 s |-| |Agent|appsec|1.202 s |137.319 ms (12.9%)| |Agent|iast|1.19 s |124.431 ms (11.7%)| |Agent|profiling|1.262 s |197.405 ms (18.5%)| |Total|tracing|10.446 s |-| |Total|appsec|10.68 s |233.828 ms (2.2%)| |Total|iast|10.812 s |366.306 ms (3.5%)| |Total|profiling|10.641 s |195.504 ms (1.9%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.072 s |-| |Agent|appsec|1.2 s |128.164 ms (12.0%)| |Agent|iast|1.208 s |135.813 ms (12.7%)| |Agent|profiling|1.27 s |197.832 ms (18.5%)| |Total|tracing|10.321 s |-| |Total|appsec|10.623 s |302.172 ms (2.9%)| |Total|iast|10.889 s |567.813 ms (5.5%)| |Total|profiling|10.663 s |342.212 ms (3.3%)| ```mermaid gantt title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~9a236deed6, baseline=1.40.0-SNAPSHOT~a369f7b401 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (679.676 ms) : 0, 679676 BytebuddyAgent [candidate] (683.753 ms) : 0, 683753 GlobalTracer [baseline] (311.185 ms) : 0, 311185 GlobalTracer [candidate] (313.538 ms) : 0, 313538 AppSec [baseline] (52.503 ms) : 0, 52503 AppSec [candidate] (52.981 ms) : 0, 52981 Remote Config [baseline] (658.921 µs) : 0, 659 Remote Config [candidate] (675.015 µs) : 0, 675 Telemetry [baseline] (7.446 ms) : 0, 7446 Telemetry [candidate] (7.56 ms) : 0, 7560 section appsec BytebuddyAgent [baseline] (702.951 ms) : 0, 702951 BytebuddyAgent [candidate] (702.268 ms) : 0, 702268 GlobalTracer [baseline] (304.043 ms) : 0, 304043 GlobalTracer [candidate] (304.233 ms) : 0, 304233 AppSec [baseline] (162.101 ms) : 0, 162101 AppSec [candidate] (160.952 ms) : 0, 160952 IAST [baseline] (19.179 ms) : 0, 19179 IAST [candidate] (21.347 ms) : 0, 21347 Remote Config [baseline] (643.25 µs) : 0, 643 Remote Config [candidate] (651.743 µs) : 0, 652 Telemetry [baseline] (9.437 ms) : 0, 9437 Telemetry [candidate] (7.992 ms) : 0, 7992 section iast BytebuddyAgent [baseline] (791.941 ms) : 0, 791941 BytebuddyAgent [candidate] (804.698 ms) : 0, 804698 GlobalTracer [baseline] (298.651 ms) : 0, 298651 GlobalTracer [candidate] (302.821 ms) : 0, 302821 AppSec [baseline] (52.859 ms) : 0, 52859 AppSec [candidate] (52.849 ms) : 0, 52849 IAST [baseline] (24.481 ms) : 0, 24481 IAST [candidate] (24.901 ms) : 0, 24901 Remote Config [baseline] (612.916 µs) : 0, 613 Remote Config [candidate] (639.648 µs) : 0, 640 Telemetry [baseline] (7.326 ms) : 0, 7326 Telemetry [candidate] (8.228 ms) : 0, 8228 section profiling BytebuddyAgent [baseline] (673.561 ms) : 0, 673561 BytebuddyAgent [candidate] (677.541 ms) : 0, 677541 GlobalTracer [baseline] (393.702 ms) : 0, 393702 GlobalTracer [candidate] (396.24 ms) : 0, 396240 AppSec [baseline] (53.149 ms) : 0, 53149 AppSec [candidate] (53.67 ms) : 0, 53670 Remote Config [baseline] (661.805 µs) : 0, 662 Remote Config [candidate] (668.124 µs) : 0, 668 Telemetry [baseline] (7.385 ms) : 0, 7385 Telemetry [candidate] (7.494 ms) : 0, 7494 ProfilingAgent [baseline] (96.066 ms) : 0, 96066 ProfilingAgent [candidate] (96.088 ms) : 0, 96088 Profiling [baseline] (96.09 ms) : 0, 96090 Profiling [candidate] (96.111 ms) : 0, 96111 ```

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-16T17:46:10 2024-09-16T17:53:01
git_branch master jpbempel/fix-flaky-smoke-tests
git_commit_date 1726504555 1726508099
git_commit_sha a369f7b401 9a236deed6
release_version 1.40.0-SNAPSHOT~a369f7b401 1.40.0-SNAPSHOT~9a236deed6
start_time 2024-09-16T17:45:56 2024-09-16T17:52:48
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1726509527 |1726509527 | |ci_job_id |641036204 |641036204 | |ci_pipeline_id|44439712 |44439712 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |iast |iast |

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 17 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-86.356µs; -35.249µs] or [-5.578%; -2.277%]		 unstable
[-316.838op/s; +790.912op/s] or [-10.693%; +26.693%]		 1.487ms 3200.000op/s 1.548ms 2962.963op/s
Request duration reports for petclinic ```mermaid gantt title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~9a236deed6, baseline=1.40.0-SNAPSHOT~a369f7b401 dateFormat X axisFormat %s section baseline no_agent (1.347 ms) : 1328, 1365 . : milestone, 1347, appsec (1.723 ms) : 1698, 1747 . : milestone, 1723, appsec_no_iast (1.728 ms) : 1704, 1753 . : milestone, 1728, iast (1.467 ms) : 1445, 1489 . : milestone, 1467, profiling (1.548 ms) : 1524, 1572 . : milestone, 1548, tracing (1.471 ms) : 1447, 1495 . : milestone, 1471, section candidate no_agent (1.343 ms) : 1324, 1363 . : milestone, 1343, appsec (1.702 ms) : 1677, 1727 . : milestone, 1702, appsec_no_iast (1.751 ms) : 1728, 1775 . : milestone, 1751, iast (1.459 ms) : 1436, 1482 . : milestone, 1459, profiling (1.487 ms) : 1464, 1510 . : milestone, 1487, tracing (1.452 ms) : 1427, 1476 . : milestone, 1452, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.347 ms [1.328 ms, 1.365 ms]|-| |appsec|1.723 ms [1.698 ms, 1.747 ms]|375.965 µs (27.9%)| |appsec_no_iast|1.728 ms [1.704 ms, 1.753 ms]|381.534 µs (28.3%)| |iast|1.467 ms [1.445 ms, 1.489 ms]|120.139 µs (8.9%)| |profiling|1.548 ms [1.524 ms, 1.572 ms]|201.328 µs (14.9%)| |tracing|1.471 ms [1.447 ms, 1.495 ms]|124.44 µs (9.2%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.343 ms [1.324 ms, 1.363 ms]|-| |appsec|1.702 ms [1.677 ms, 1.727 ms]|358.872 µs (26.7%)| |appsec_no_iast|1.751 ms [1.728 ms, 1.775 ms]|408.073 µs (30.4%)| |iast|1.459 ms [1.436 ms, 1.482 ms]|116.313 µs (8.7%)| |profiling|1.487 ms [1.464 ms, 1.51 ms]|144.198 µs (10.7%)| |tracing|1.452 ms [1.427 ms, 1.476 ms]|108.403 µs (8.1%)|
Request duration reports for insecure-bank ```mermaid gantt title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~9a236deed6, baseline=1.40.0-SNAPSHOT~a369f7b401 dateFormat X axisFormat %s section baseline no_agent (371.705 µs) : 352, 391 . : milestone, 372, iast (481.297 µs) : 460, 503 . : milestone, 481, iast_FULL (552.763 µs) : 532, 574 . : milestone, 553, iast_GLOBAL (514.775 µs) : 492, 537 . : milestone, 515, iast_HARDCODED_SECRET_DISABLED (493.448 µs) : 472, 515 . : milestone, 493, iast_INACTIVE (447.427 µs) : 426, 469 . : milestone, 447, iast_TELEMETRY_OFF (475.543 µs) : 453, 498 . : milestone, 476, tracing (442.764 µs) : 422, 463 . : milestone, 443, section candidate no_agent (362.721 µs) : 343, 382 . : milestone, 363, iast (491.421 µs) : 470, 513 . : milestone, 491, iast_FULL (557.117 µs) : 536, 578 . : milestone, 557, iast_GLOBAL (504.991 µs) : 484, 526 . : milestone, 505, iast_HARDCODED_SECRET_DISABLED (479.258 µs) : 459, 500 . : milestone, 479, iast_INACTIVE (450.063 µs) : 429, 471 . : milestone, 450, iast_TELEMETRY_OFF (478.764 µs) : 455, 502 . : milestone, 479, tracing (449.202 µs) : 429, 470 . : milestone, 449, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|371.705 µs [352.075 µs, 391.336 µs]|-| |iast|481.297 µs [459.675 µs, 502.919 µs]|109.591 µs (29.5%)| |iast_FULL|552.763 µs [531.653 µs, 573.872 µs]|181.058 µs (48.7%)| |iast_GLOBAL|514.775 µs [492.141 µs, 537.408 µs]|143.069 µs (38.5%)| |iast_HARDCODED_SECRET_DISABLED|493.448 µs [471.981 µs, 514.914 µs]|121.742 µs (32.8%)| |iast_INACTIVE|447.427 µs [426.304 µs, 468.549 µs]|75.721 µs (20.4%)| |iast_TELEMETRY_OFF|475.543 µs [453.298 µs, 497.787 µs]|103.837 µs (27.9%)| |tracing|442.764 µs [422.411 µs, 463.117 µs]|71.059 µs (19.1%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|362.721 µs [342.957 µs, 382.485 µs]|-| |iast|491.421 µs [470.217 µs, 512.625 µs]|128.7 µs (35.5%)| |iast_FULL|557.117 µs [535.881 µs, 578.354 µs]|194.396 µs (53.6%)| |iast_GLOBAL|504.991 µs [484.042 µs, 525.939 µs]|142.27 µs (39.2%)| |iast_HARDCODED_SECRET_DISABLED|479.258 µs [458.501 µs, 500.015 µs]|116.537 µs (32.1%)| |iast_INACTIVE|450.063 µs [428.816 µs, 471.311 µs]|87.342 µs (24.1%)| |iast_TELEMETRY_OFF|478.764 µs [455.455 µs, 502.074 µs]|116.043 µs (32.0%)| |tracing|449.202 µs [428.672 µs, 469.733 µs]|86.481 µs (23.8%)|

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-smoke-tests
git_commit_date 1726504555 1726508099
git_commit_sha a369f7b401 9a236deed6
release_version 1.40.0-SNAPSHOT~a369f7b401 1.40.0-SNAPSHOT~9a236deed6
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |biojava |biojava | |ci_job_date |1726510048 |1726510048 | |ci_job_id |641036205 |641036205 | |ci_pipeline_id|44439712 |44439712 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |appsec |appsec |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat ```mermaid gantt title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~9a236deed6, baseline=1.40.0-SNAPSHOT~a369f7b401 dateFormat X axisFormat %s section baseline no_agent (1.469 ms) : 1457, 1480 . : milestone, 1469, appsec (2.286 ms) : 2246, 2326 . : milestone, 2286, iast (2.059 ms) : 2009, 2109 . : milestone, 2059, iast_GLOBAL (2.087 ms) : 2038, 2137 . : milestone, 2087, profiling (1.925 ms) : 1886, 1965 . : milestone, 1925, tracing (1.907 ms) : 1869, 1945 . : milestone, 1907, section candidate no_agent (1.464 ms) : 1452, 1475 . : milestone, 1464, appsec (2.291 ms) : 2250, 2331 . : milestone, 2291, iast (2.053 ms) : 2003, 2103 . : milestone, 2053, iast_GLOBAL (2.106 ms) : 2055, 2156 . : milestone, 2106, profiling (1.929 ms) : 1888, 1970 . : milestone, 1929, tracing (1.893 ms) : 1855, 1930 . : milestone, 1893, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.469 ms [1.457 ms, 1.48 ms]|-| |appsec|2.286 ms [2.246 ms, 2.326 ms]|817.45 µs (55.7%)| |iast|2.059 ms [2.009 ms, 2.109 ms]|590.465 µs (40.2%)| |iast_GLOBAL|2.087 ms [2.038 ms, 2.137 ms]|618.933 µs (42.1%)| |profiling|1.925 ms [1.886 ms, 1.965 ms]|456.855 µs (31.1%)| |tracing|1.907 ms [1.869 ms, 1.945 ms]|438.609 µs (29.9%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.464 ms [1.452 ms, 1.475 ms]|-| |appsec|2.291 ms [2.25 ms, 2.331 ms]|826.661 µs (56.5%)| |iast|2.053 ms [2.003 ms, 2.103 ms]|588.905 µs (40.2%)| |iast_GLOBAL|2.106 ms [2.055 ms, 2.156 ms]|641.873 µs (43.8%)| |profiling|1.929 ms [1.888 ms, 1.97 ms]|464.603 µs (31.7%)| |tracing|1.893 ms [1.855 ms, 1.93 ms]|428.956 µs (29.3%)|
Execution time for biojava ```mermaid gantt title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~9a236deed6, baseline=1.40.0-SNAPSHOT~a369f7b401 dateFormat X axisFormat %s section baseline no_agent (15.311 s) : 15311000, 15311000 . : milestone, 15311000, appsec (15.154 s) : 15154000, 15154000 . : milestone, 15154000, iast (19.191 s) : 19191000, 19191000 . : milestone, 19191000, iast_GLOBAL (18.198 s) : 18198000, 18198000 . : milestone, 18198000, profiling (15.741 s) : 15741000, 15741000 . : milestone, 15741000, tracing (15.332 s) : 15332000, 15332000 . : milestone, 15332000, section candidate no_agent (15.037 s) : 15037000, 15037000 . : milestone, 15037000, appsec (15.247 s) : 15247000, 15247000 . : milestone, 15247000, iast (19.128 s) : 19128000, 19128000 . : milestone, 19128000, iast_GLOBAL (18.08 s) : 18080000, 18080000 . : milestone, 18080000, profiling (15.987 s) : 15987000, 15987000 . : milestone, 15987000, tracing (15.182 s) : 15182000, 15182000 . : milestone, 15182000, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.311 s [15.311 s, 15.311 s]|-| |appsec|15.154 s [15.154 s, 15.154 s]|-157.0 ms (-1.0%)| |iast|19.191 s [19.191 s, 19.191 s]|3.88 s (25.3%)| |iast_GLOBAL|18.198 s [18.198 s, 18.198 s]|2.887 s (18.9%)| |profiling|15.741 s [15.741 s, 15.741 s]|430.0 ms (2.8%)| |tracing|15.332 s [15.332 s, 15.332 s]|21.0 ms (0.1%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.037 s [15.037 s, 15.037 s]|-| |appsec|15.247 s [15.247 s, 15.247 s]|210.0 ms (1.4%)| |iast|19.128 s [19.128 s, 19.128 s]|4.091 s (27.2%)| |iast_GLOBAL|18.08 s [18.08 s, 18.08 s]|3.043 s (20.2%)| |profiling|15.987 s [15.987 s, 15.987 s]|950.0 ms (6.3%)| |tracing|15.182 s [15.182 s, 15.182 s]|145.0 ms (1.0%)|