search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
572 stars 284 forks source link

Fix evaluation errors sampling #7656

Closed jpbempel closed 1 week ago

jpbempel commented 1 week ago

What Does This Do

evaluation error snapshots were sent even if rate limited now we are checking sampled state before sending

Motivation

Additional Notes

Contributor Checklist

Jira ticket: DEBUG-2892

pr-commenter[bot] commented 1 week ago

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1726837217 1726837593
end_time 2024-09-20T13:01:32 2024-09-20T13:07:48
git_branch master jpbempel/fix-eval-errors-sampling
git_commit_sha 58679718b3 ac44f215cf4a9b1281991c98cc1ae058f7bc05ce
start_time 2024-09-20T13:00:18 2024-09-20T13:06:33
See matching parameters | |Baseline |Candidate | |----------------------|----------------------------------------------|----------------------------------------------| |ci_job_id |646380732 |646380732 | |ci_pipeline_id |44786335 |44786335 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |git_commit_date|1726836409 |1726836409 |

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 2 metrics, 8 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+670.825µs; +700.305µs] or [+6.617%; +6.908%]		 worse
[+666.425µs; +722.055µs] or [+6.474%; +7.015%]		 worse
[+595.043µs; +706.833µs] or [+5.716%; +6.790%]		 worse
[+422.099µs; +855.145µs] or [+3.909%; +7.919%]		 worse
[-7.390op/s; -4.625op/s] or [-7.722%; -4.834%]
See unchanged results |scenario |Δ mean agg_http_req_duration_min |Δ mean agg_http_req_duration_p50 |Δ mean agg_http_req_duration_p75 |Δ mean agg_http_req_duration_p99 |Δ mean throughput | |----------------|-----------------------------------------------------------------|-----------------------------------------------------------------|-----------------------------------------------------------------|--------------------------------------------------------------------|---------------------------------------------------------------------| |scenario:noprobe|unstable
[-51.766µs; +62.290µs] or [-18.881%; +22.719%]|unstable
[-61.710µs; +78.989µs] or [-19.585%; +25.069%]|unstable
[-70.852µs; +91.457µs] or [-21.464%; +27.706%]|unstable
[-796.903µs; +883.967µs] or [-93.514%; +103.730%]|same | |scenario:basic |same |unstable
[-14.890µs; +19.922µs] or [-4.957%; +6.632%] |unstable
[-19.023µs; +27.190µs] or [-6.124%; +8.753%] |unstable
[-104.475µs; +152.020µs] or [-18.976%; +27.611%] |unstable
[-236.857op/s; +236.857op/s] or [-9.237%; +9.237%]|
Request duration reports for reports ```mermaid gantt title reports - request duration [CI 0.99] : candidate=None, baseline=None dateFormat X axisFormat %s section baseline noprobe (315.087 µs) : 252, 378 . : milestone, 315, basic (300.378 µs) : 283, 317 . : milestone, 300, loop (10.294 ms) : 10265, 10322 . : milestone, 10294, section candidate noprobe (323.727 µs) : 253, 394 . : milestone, 324, basic (302.894 µs) : 286, 320 . : milestone, 303, loop (10.988 ms) : 10965, 11011 . : milestone, 10988, ``` * **baseline** results |Scenario|Request median duration [CI 0.99]| |---|---| |noprobe|315.087 µs [251.795 µs, 378.38 µs]| |basic|300.378 µs [283.462 µs, 317.293 µs]| |loop|10.294 ms [10.265 ms, 10.322 ms]| * **candidate** results |Scenario|Request median duration [CI 0.99]| |---|---| |noprobe|323.727 µs [253.042 µs, 394.412 µs]| |basic|302.894 µs [285.755 µs, 320.033 µs]| |loop|10.988 ms [10.965 ms, 11.011 ms]|
pr-commenter[bot] commented 1 week ago

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-eval-errors-sampling
git_commit_date 1726823241 1726836409
git_commit_sha 58679718b3 ac44f215cf
release_version 1.40.0-SNAPSHOT~58679718b3 1.40.0-SNAPSHOT~ac44f215cf
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1726838978 |1726838978 | |ci_job_id |646380726 |646380726 | |ci_pipeline_id|44786335 |44786335 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |module |Agent |Agent | |parent |None |None | |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for petclinic ```mermaid gantt title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~ac44f215cf, baseline=1.40.0-SNAPSHOT~58679718b3 dateFormat X axisFormat %s section tracing Agent [baseline] (1.066 s) : 0, 1065876 Total [baseline] (10.411 s) : 0, 10410874 Agent [candidate] (1.067 s) : 0, 1066929 Total [candidate] (10.422 s) : 0, 10421933 section appsec Agent [baseline] (1.197 s) : 0, 1196586 Total [baseline] (10.653 s) : 0, 10652517 Agent [candidate] (1.199 s) : 0, 1198824 Total [candidate] (10.602 s) : 0, 10601576 section iast Agent [baseline] (1.196 s) : 0, 1196131 Total [baseline] (10.812 s) : 0, 10811791 Agent [candidate] (1.19 s) : 0, 1189992 Total [candidate] (10.878 s) : 0, 10878424 section profiling Agent [baseline] (1.269 s) : 0, 1269161 Total [baseline] (10.542 s) : 0, 10541858 Agent [candidate] (1.271 s) : 0, 1271297 Total [candidate] (10.566 s) : 0, 10566462 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.066 s |-| |Agent|appsec|1.197 s |130.711 ms (12.3%)| |Agent|iast|1.196 s |130.255 ms (12.2%)| |Agent|profiling|1.269 s |203.286 ms (19.1%)| |Total|tracing|10.411 s |-| |Total|appsec|10.653 s |241.643 ms (2.3%)| |Total|iast|10.812 s |400.917 ms (3.9%)| |Total|profiling|10.542 s |130.984 ms (1.3%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.067 s |-| |Agent|appsec|1.199 s |131.895 ms (12.4%)| |Agent|iast|1.19 s |123.062 ms (11.5%)| |Agent|profiling|1.271 s |204.368 ms (19.2%)| |Total|tracing|10.422 s |-| |Total|appsec|10.602 s |179.644 ms (1.7%)| |Total|iast|10.878 s |456.491 ms (4.4%)| |Total|profiling|10.566 s |144.529 ms (1.4%)| ```mermaid gantt title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~ac44f215cf, baseline=1.40.0-SNAPSHOT~58679718b3 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (678.37 ms) : 0, 678370 BytebuddyAgent [candidate] (679.043 ms) : 0, 679043 GlobalTracer [baseline] (311.731 ms) : 0, 311731 GlobalTracer [candidate] (311.658 ms) : 0, 311658 AppSec [baseline] (53.881 ms) : 0, 53881 AppSec [candidate] (54.223 ms) : 0, 54223 Remote Config [baseline] (665.375 µs) : 0, 665 Remote Config [candidate] (682.183 µs) : 0, 682 Telemetry [baseline] (7.645 ms) : 0, 7645 Telemetry [candidate] (7.691 ms) : 0, 7691 section appsec BytebuddyAgent [baseline] (700.202 ms) : 0, 700202 BytebuddyAgent [candidate] (700.26 ms) : 0, 700260 GlobalTracer [baseline] (302.387 ms) : 0, 302387 GlobalTracer [candidate] (302.385 ms) : 0, 302385 AppSec [baseline] (161.451 ms) : 0, 161451 AppSec [candidate] (162.608 ms) : 0, 162608 Remote Config [baseline] (642.041 µs) : 0, 642 Remote Config [candidate] (636.969 µs) : 0, 637 Telemetry [baseline] (8.06 ms) : 0, 8060 Telemetry [candidate] (8.815 ms) : 0, 8815 IAST [baseline] (20.587 ms) : 0, 20587 IAST [candidate] (21.118 ms) : 0, 21118 section iast BytebuddyAgent [baseline] (794.737 ms) : 0, 794737 BytebuddyAgent [candidate] (791.651 ms) : 0, 791651 GlobalTracer [baseline] (300.607 ms) : 0, 300607 GlobalTracer [candidate] (298.679 ms) : 0, 298679 AppSec [baseline] (53.582 ms) : 0, 53582 AppSec [candidate] (53.337 ms) : 0, 53337 Remote Config [baseline] (625.449 µs) : 0, 625 Remote Config [candidate] (658.767 µs) : 0, 659 Telemetry [baseline] (7.4 ms) : 0, 7400 Telemetry [candidate] (7.388 ms) : 0, 7388 IAST [baseline] (25.534 ms) : 0, 25534 IAST [candidate] (24.653 ms) : 0, 24653 section profiling ProfilingAgent [baseline] (95.773 ms) : 0, 95773 ProfilingAgent [candidate] (96.856 ms) : 0, 96856 BytebuddyAgent [baseline] (677.117 ms) : 0, 677117 BytebuddyAgent [candidate] (676.482 ms) : 0, 676482 GlobalTracer [baseline] (395.271 ms) : 0, 395271 GlobalTracer [candidate] (396.835 ms) : 0, 396835 AppSec [baseline] (54.587 ms) : 0, 54587 AppSec [candidate] (54.78 ms) : 0, 54780 Remote Config [baseline] (657.024 µs) : 0, 657 Remote Config [candidate] (664.5 µs) : 0, 665 Telemetry [baseline] (7.464 ms) : 0, 7464 Telemetry [candidate] (7.475 ms) : 0, 7475 Profiling [baseline] (95.796 ms) : 0, 95796 Profiling [candidate] (96.881 ms) : 0, 96881 ```
Startup time reports for insecure-bank ```mermaid gantt title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~ac44f215cf, baseline=1.40.0-SNAPSHOT~58679718b3 dateFormat X axisFormat %s section tracing Agent [baseline] (1.063 s) : 0, 1062941 Total [baseline] (8.508 s) : 0, 8507547 Agent [candidate] (1.064 s) : 0, 1064285 Total [candidate] (8.526 s) : 0, 8525592 section iast Agent [baseline] (1.188 s) : 0, 1187923 Total [baseline] (8.993 s) : 0, 8992690 Agent [candidate] (1.206 s) : 0, 1206359 Total [candidate] (9.003 s) : 0, 9003166 section iast_HARDCODED_SECRET_DISABLED Agent [baseline] (1.188 s) : 0, 1188346 Total [baseline] (8.949 s) : 0, 8949283 Agent [candidate] (1.188 s) : 0, 1188168 Total [candidate] (9.001 s) : 0, 9000652 section iast_TELEMETRY_OFF Agent [baseline] (1.194 s) : 0, 1194038 Total [baseline] (9.045 s) : 0, 9044946 Agent [candidate] (1.186 s) : 0, 1186198 Total [candidate] (8.989 s) : 0, 8989087 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.063 s |-| |Agent|iast|1.188 s |124.982 ms (11.8%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.188 s |125.405 ms (11.8%)| |Agent|iast_TELEMETRY_OFF|1.194 s |131.098 ms (12.3%)| |Total|tracing|8.508 s |-| |Total|iast|8.993 s |485.143 ms (5.7%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.949 s |441.735 ms (5.2%)| |Total|iast_TELEMETRY_OFF|9.045 s |537.399 ms (6.3%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.064 s |-| |Agent|iast|1.206 s |142.074 ms (13.3%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.188 s |123.883 ms (11.6%)| |Agent|iast_TELEMETRY_OFF|1.186 s |121.913 ms (11.5%)| |Total|tracing|8.526 s |-| |Total|iast|9.003 s |477.574 ms (5.6%)| |Total|iast_HARDCODED_SECRET_DISABLED|9.001 s |475.06 ms (5.6%)| |Total|iast_TELEMETRY_OFF|8.989 s |463.496 ms (5.4%)| ```mermaid gantt title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~ac44f215cf, baseline=1.40.0-SNAPSHOT~58679718b3 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (677.964 ms) : 0, 677964 BytebuddyAgent [candidate] (678.783 ms) : 0, 678783 GlobalTracer [baseline] (309.635 ms) : 0, 309635 GlobalTracer [candidate] (310.087 ms) : 0, 310087 AppSec [baseline] (53.507 ms) : 0, 53507 AppSec [candidate] (53.565 ms) : 0, 53565 Remote Config [baseline] (671.854 µs) : 0, 672 Remote Config [candidate] (665.269 µs) : 0, 665 Telemetry [baseline] (7.58 ms) : 0, 7580 Telemetry [candidate] (7.557 ms) : 0, 7557 section iast BytebuddyAgent [baseline] (790.971 ms) : 0, 790971 BytebuddyAgent [candidate] (803.019 ms) : 0, 803019 GlobalTracer [baseline] (297.501 ms) : 0, 297501 GlobalTracer [candidate] (302.636 ms) : 0, 302636 AppSec [baseline] (53.77 ms) : 0, 53770 AppSec [candidate] (57.072 ms) : 0, 57072 Remote Config [baseline] (629.793 µs) : 0, 630 Remote Config [candidate] (604.215 µs) : 0, 604 Telemetry [baseline] (7.279 ms) : 0, 7279 Telemetry [candidate] (7.286 ms) : 0, 7286 IAST [baseline] (24.162 ms) : 0, 24162 IAST [candidate] (21.883 ms) : 0, 21883 section iast_HARDCODED_SECRET_DISABLED BytebuddyAgent [baseline] (791.153 ms) : 0, 791153 BytebuddyAgent [candidate] (790.025 ms) : 0, 790025 GlobalTracer [baseline] (298.069 ms) : 0, 298069 GlobalTracer [candidate] (298.112 ms) : 0, 298112 AppSec [baseline] (53.955 ms) : 0, 53955 AppSec [candidate] (54.97 ms) : 0, 54970 Remote Config [baseline] (618.742 µs) : 0, 619 Remote Config [candidate] (612.185 µs) : 0, 612 Telemetry [baseline] (7.333 ms) : 0, 7333 Telemetry [candidate] (7.322 ms) : 0, 7322 IAST [baseline] (23.561 ms) : 0, 23561 IAST [candidate] (23.49 ms) : 0, 23490 section iast_TELEMETRY_OFF BytebuddyAgent [baseline] (793.503 ms) : 0, 793503 BytebuddyAgent [candidate] (787.753 ms) : 0, 787753 GlobalTracer [baseline] (300.282 ms) : 0, 300282 GlobalTracer [candidate] (298.694 ms) : 0, 298694 AppSec [baseline] (52.713 ms) : 0, 52713 AppSec [candidate] (53.612 ms) : 0, 53612 Remote Config [baseline] (1.369 ms) : 0, 1369 Remote Config [candidate] (595.963 µs) : 0, 596 Telemetry [baseline] (7.294 ms) : 0, 7294 Telemetry [candidate] (7.132 ms) : 0, 7132 IAST [baseline] (25.152 ms) : 0, 25152 IAST [candidate] (24.786 ms) : 0, 24786 ```

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-20T13:00:15 2024-09-20T13:07:04
git_branch master jpbempel/fix-eval-errors-sampling
git_commit_date 1726823241 1726836409
git_commit_sha 58679718b3 ac44f215cf
release_version 1.40.0-SNAPSHOT~58679718b3 1.40.0-SNAPSHOT~ac44f215cf
start_time 2024-09-20T13:00:02 2024-09-20T13:06:51
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1726837970 |1726837970 | |ci_job_id |646380727 |646380727 | |ci_pipeline_id|44786335 |44786335 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics.

Request duration reports for petclinic ```mermaid gantt title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~ac44f215cf, baseline=1.40.0-SNAPSHOT~58679718b3 dateFormat X axisFormat %s section baseline no_agent (1.348 ms) : 1328, 1368 . : milestone, 1348, appsec (1.724 ms) : 1701, 1748 . : milestone, 1724, appsec_no_iast (1.697 ms) : 1672, 1722 . : milestone, 1697, iast (1.47 ms) : 1447, 1493 . : milestone, 1470, profiling (1.483 ms) : 1458, 1508 . : milestone, 1483, tracing (1.46 ms) : 1436, 1484 . : milestone, 1460, section candidate no_agent (1.319 ms) : 1300, 1339 . : milestone, 1319, appsec (1.721 ms) : 1697, 1744 . : milestone, 1721, appsec_no_iast (1.725 ms) : 1700, 1749 . : milestone, 1725, iast (1.478 ms) : 1455, 1501 . : milestone, 1478, profiling (1.5 ms) : 1474, 1526 . : milestone, 1500, tracing (1.482 ms) : 1458, 1507 . : milestone, 1482, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.348 ms [1.328 ms, 1.368 ms]|-| |appsec|1.724 ms [1.701 ms, 1.748 ms]|376.202 µs (27.9%)| |appsec_no_iast|1.697 ms [1.672 ms, 1.722 ms]|348.931 µs (25.9%)| |iast|1.47 ms [1.447 ms, 1.493 ms]|122.008 µs (9.1%)| |profiling|1.483 ms [1.458 ms, 1.508 ms]|135.292 µs (10.0%)| |tracing|1.46 ms [1.436 ms, 1.484 ms]|112.265 µs (8.3%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.319 ms [1.3 ms, 1.339 ms]|-| |appsec|1.721 ms [1.697 ms, 1.744 ms]|401.812 µs (30.5%)| |appsec_no_iast|1.725 ms [1.7 ms, 1.749 ms]|405.75 µs (30.8%)| |iast|1.478 ms [1.455 ms, 1.501 ms]|158.647 µs (12.0%)| |profiling|1.5 ms [1.474 ms, 1.526 ms]|180.608 µs (13.7%)| |tracing|1.482 ms [1.458 ms, 1.507 ms]|163.297 µs (12.4%)|
Request duration reports for insecure-bank ```mermaid gantt title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~ac44f215cf, baseline=1.40.0-SNAPSHOT~58679718b3 dateFormat X axisFormat %s section baseline no_agent (365.293 µs) : 345, 385 . : milestone, 365, iast (478.803 µs) : 458, 500 . : milestone, 479, iast_FULL (549.392 µs) : 528, 571 . : milestone, 549, iast_GLOBAL (504.685 µs) : 483, 526 . : milestone, 505, iast_HARDCODED_SECRET_DISABLED (478.902 µs) : 458, 500 . : milestone, 479, iast_INACTIVE (447.77 µs) : 426, 469 . : milestone, 448, iast_TELEMETRY_OFF (466.417 µs) : 444, 489 . : milestone, 466, tracing (436.105 µs) : 416, 457 . : milestone, 436, section candidate no_agent (373.579 µs) : 354, 393 . : milestone, 374, iast (481.022 µs) : 459, 503 . : milestone, 481, iast_FULL (543.825 µs) : 522, 565 . : milestone, 544, iast_GLOBAL (500.644 µs) : 479, 523 . : milestone, 501, iast_HARDCODED_SECRET_DISABLED (475.967 µs) : 455, 497 . : milestone, 476, iast_INACTIVE (441.815 µs) : 421, 463 . : milestone, 442, iast_TELEMETRY_OFF (473.07 µs) : 451, 496 . : milestone, 473, tracing (441.167 µs) : 421, 462 . : milestone, 441, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|365.293 µs [345.126 µs, 385.459 µs]|-| |iast|478.803 µs [457.727 µs, 499.88 µs]|113.511 µs (31.1%)| |iast_FULL|549.392 µs [528.114 µs, 570.67 µs]|184.099 µs (50.4%)| |iast_GLOBAL|504.685 µs [482.952 µs, 526.419 µs]|139.393 µs (38.2%)| |iast_HARDCODED_SECRET_DISABLED|478.902 µs [457.675 µs, 500.13 µs]|113.61 µs (31.1%)| |iast_INACTIVE|447.77 µs [426.229 µs, 469.31 µs]|82.477 µs (22.6%)| |iast_TELEMETRY_OFF|466.417 µs [443.865 µs, 488.968 µs]|101.124 µs (27.7%)| |tracing|436.105 µs [415.585 µs, 456.625 µs]|70.812 µs (19.4%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|373.579 µs [353.993 µs, 393.166 µs]|-| |iast|481.022 µs [459.374 µs, 502.67 µs]|107.443 µs (28.8%)| |iast_FULL|543.825 µs [522.42 µs, 565.23 µs]|170.245 µs (45.6%)| |iast_GLOBAL|500.644 µs [478.716 µs, 522.573 µs]|127.065 µs (34.0%)| |iast_HARDCODED_SECRET_DISABLED|475.967 µs [454.983 µs, 496.95 µs]|102.387 µs (27.4%)| |iast_INACTIVE|441.815 µs [420.751 µs, 462.878 µs]|68.235 µs (18.3%)| |iast_TELEMETRY_OFF|473.07 µs [450.562 µs, 495.577 µs]|99.49 µs (26.6%)| |tracing|441.167 µs [420.652 µs, 461.683 µs]|67.588 µs (18.1%)|

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-eval-errors-sampling
git_commit_date 1726823241 1726836409
git_commit_sha 58679718b3 ac44f215cf
release_version 1.40.0-SNAPSHOT~58679718b3 1.40.0-SNAPSHOT~ac44f215cf
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |biojava |biojava | |ci_job_date |1726838531 |1726838531 | |ci_job_id |646380728 |646380728 | |ci_pipeline_id|44786335 |44786335 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |appsec |appsec |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat ```mermaid gantt title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~ac44f215cf, baseline=1.40.0-SNAPSHOT~58679718b3 dateFormat X axisFormat %s section baseline no_agent (1.47 ms) : 1459, 1482 . : milestone, 1470, appsec (2.329 ms) : 2289, 2370 . : milestone, 2329, iast (2.053 ms) : 2004, 2102 . : milestone, 2053, iast_GLOBAL (2.097 ms) : 2046, 2147 . : milestone, 2097, profiling (1.928 ms) : 1889, 1968 . : milestone, 1928, tracing (1.899 ms) : 1861, 1936 . : milestone, 1899, section candidate no_agent (1.47 ms) : 1459, 1482 . : milestone, 1470, appsec (2.317 ms) : 2276, 2357 . : milestone, 2317, iast (2.063 ms) : 2013, 2113 . : milestone, 2063, iast_GLOBAL (2.11 ms) : 2060, 2160 . : milestone, 2110, profiling (1.932 ms) : 1891, 1972 . : milestone, 1932, tracing (1.898 ms) : 1859, 1936 . : milestone, 1898, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.47 ms [1.459 ms, 1.482 ms]|-| |appsec|2.329 ms [2.289 ms, 2.37 ms]|859.283 µs (58.4%)| |iast|2.053 ms [2.004 ms, 2.102 ms]|582.729 µs (39.6%)| |iast_GLOBAL|2.097 ms [2.046 ms, 2.147 ms]|626.539 µs (42.6%)| |profiling|1.928 ms [1.889 ms, 1.968 ms]|458.201 µs (31.2%)| |tracing|1.899 ms [1.861 ms, 1.936 ms]|428.565 µs (29.2%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.47 ms [1.459 ms, 1.482 ms]|-| |appsec|2.317 ms [2.276 ms, 2.357 ms]|846.378 µs (57.6%)| |iast|2.063 ms [2.013 ms, 2.113 ms]|592.857 µs (40.3%)| |iast_GLOBAL|2.11 ms [2.06 ms, 2.16 ms]|639.416 µs (43.5%)| |profiling|1.932 ms [1.891 ms, 1.972 ms]|461.532 µs (31.4%)| |tracing|1.898 ms [1.859 ms, 1.936 ms]|427.213 µs (29.1%)|
Execution time for biojava ```mermaid gantt title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~ac44f215cf, baseline=1.40.0-SNAPSHOT~58679718b3 dateFormat X axisFormat %s section baseline no_agent (14.928 s) : 14928000, 14928000 . : milestone, 14928000, appsec (15.185 s) : 15185000, 15185000 . : milestone, 15185000, iast (18.996 s) : 18996000, 18996000 . : milestone, 18996000, iast_GLOBAL (17.811 s) : 17811000, 17811000 . : milestone, 17811000, profiling (15.036 s) : 15036000, 15036000 . : milestone, 15036000, tracing (15.016 s) : 15016000, 15016000 . : milestone, 15016000, section candidate no_agent (14.897 s) : 14897000, 14897000 . : milestone, 14897000, appsec (15.231 s) : 15231000, 15231000 . : milestone, 15231000, iast (18.959 s) : 18959000, 18959000 . : milestone, 18959000, iast_GLOBAL (17.908 s) : 17908000, 17908000 . : milestone, 17908000, profiling (15.139 s) : 15139000, 15139000 . : milestone, 15139000, tracing (15.136 s) : 15136000, 15136000 . : milestone, 15136000, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|14.928 s [14.928 s, 14.928 s]|-| |appsec|15.185 s [15.185 s, 15.185 s]|257.0 ms (1.7%)| |iast|18.996 s [18.996 s, 18.996 s]|4.068 s (27.3%)| |iast_GLOBAL|17.811 s [17.811 s, 17.811 s]|2.883 s (19.3%)| |profiling|15.036 s [15.036 s, 15.036 s]|108.0 ms (0.7%)| |tracing|15.016 s [15.016 s, 15.016 s]|88.0 ms (0.6%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|14.897 s [14.897 s, 14.897 s]|-| |appsec|15.231 s [15.231 s, 15.231 s]|334.0 ms (2.2%)| |iast|18.959 s [18.959 s, 18.959 s]|4.062 s (27.3%)| |iast_GLOBAL|17.908 s [17.908 s, 17.908 s]|3.011 s (20.2%)| |profiling|15.139 s [15.139 s, 15.139 s]|242.0 ms (1.6%)| |tracing|15.136 s [15.136 s, 15.136 s]|239.0 ms (1.6%)|