search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
573 stars 284 forks source link

Disabled test for local vars on IBM JDK #7577

Closed jpbempel closed 3 weeks ago

jpbempel commented 3 weeks ago

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

pr-commenter[bot] commented 3 weeks ago

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1725886234 1725886609
end_time 2024-09-09T12:51:47 2024-09-09T12:58:03
git_branch master jpbempel/fix-flaky-ibm
git_commit_sha 5ddb19db3a d416dc519f5e8c90ae2ce40e7ce384e7e95be065
start_time 2024-09-09T12:50:35 2024-09-09T12:56:50
See matching parameters | |Baseline |Candidate | |----------------------|----------------------------------------------|----------------------------------------------| |ci_job_id |631837483 |631837483 | |ci_pipeline_id |43848836 |43848836 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |git_commit_date|1725885618 |1725885618 |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 5 unstable metrics.

See unchanged results |scenario |Δ mean agg_http_req_duration_min |Δ mean agg_http_req_duration_p50 |Δ mean agg_http_req_duration_p75 |Δ mean agg_http_req_duration_p99 |Δ mean throughput| |----------------|---------------------------------------------------------------------|----------------------------------------------------------------|-----------------------------------------------------------------|-----------------------------------------------------------------|-----------------| |scenario:noprobe|unstable
[-13742.456ns; +15633.357ns] or [-5.491%; +6.246%]|unstable
[-20.473µs; +29.486µs] or [-7.206%; +10.378%]|unstable
[-30.712µs; +40.164µs] or [-10.341%; +13.524%]|unstable
[-149.456µs; +60.992µs] or [-21.905%; +8.940%]|same | |scenario:basic |same |same |same |unstable
[-6.146µs; +100.698µs] or [-1.065%; +17.448%] |same | |scenario:loop |unsure
[-56.681µs; -43.918µs] or [-0.559%; -0.433%] |unsure
[-66.455µs; -17.359µs] or [-0.645%; -0.169%] |same |same |same |
Request duration reports for reports ```mermaid gantt title reports - request duration [CI 0.99] : candidate=None, baseline=None dateFormat X axisFormat %s section baseline noprobe (284.119 µs) : 259, 309 . : milestone, 284, basic (287.709 µs) : 279, 296 . : milestone, 288, loop (10.296 ms) : 10285, 10308 . : milestone, 10296, section candidate noprobe (288.625 µs) : 266, 311 . : milestone, 289, basic (291.179 µs) : 284, 298 . : milestone, 291, loop (10.255 ms) : 10224, 10285 . : milestone, 10255, ``` * **baseline** results |Scenario|Request median duration [CI 0.99]| |---|---| |noprobe|284.119 µs [258.883 µs, 309.355 µs]| |basic|287.709 µs [279.178 µs, 296.239 µs]| |loop|10.296 ms [10.285 ms, 10.308 ms]| * **candidate** results |Scenario|Request median duration [CI 0.99]| |---|---| |noprobe|288.625 µs [266.268 µs, 310.983 µs]| |basic|291.179 µs [283.899 µs, 298.46 µs]| |loop|10.255 ms [10.224 ms, 10.285 ms]|
pr-commenter[bot] commented 3 weeks ago

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-ibm
git_commit_date 1725881796 1725884395
git_commit_sha c4dfa36730 942b1e655b
release_version 1.40.0-SNAPSHOT~c4dfa36730 1.40.0-SNAPSHOT~942b1e655b
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1725886749 |1725886749 | |ci_job_id |631799208 |631799208 | |ci_pipeline_id|43844844 |43844844 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |module |Agent |Agent | |parent |None |None | |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for insecure-bank ```mermaid gantt title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~942b1e655b, baseline=1.40.0-SNAPSHOT~c4dfa36730 dateFormat X axisFormat %s section tracing Agent [baseline] (1.051 s) : 0, 1051140 Total [baseline] (8.495 s) : 0, 8495279 Agent [candidate] (1.051 s) : 0, 1050951 Total [candidate] (8.498 s) : 0, 8497965 section iast Agent [baseline] (1.184 s) : 0, 1184333 Total [baseline] (8.978 s) : 0, 8978486 Agent [candidate] (1.175 s) : 0, 1175209 Total [candidate] (8.978 s) : 0, 8978380 section iast_HARDCODED_SECRET_DISABLED Agent [baseline] (1.175 s) : 0, 1175211 Total [baseline] (8.922 s) : 0, 8921977 Agent [candidate] (1.184 s) : 0, 1183784 Total [candidate] (8.959 s) : 0, 8958786 section iast_TELEMETRY_OFF Agent [baseline] (1.178 s) : 0, 1177816 Total [baseline] (8.971 s) : 0, 8971145 Agent [candidate] (1.172 s) : 0, 1172475 Total [candidate] (8.986 s) : 0, 8985996 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.051 s |-| |Agent|iast|1.184 s |133.193 ms (12.7%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.175 s |124.072 ms (11.8%)| |Agent|iast_TELEMETRY_OFF|1.178 s |126.676 ms (12.1%)| |Total|tracing|8.495 s |-| |Total|iast|8.978 s |483.207 ms (5.7%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.922 s |426.698 ms (5.0%)| |Total|iast_TELEMETRY_OFF|8.971 s |475.866 ms (5.6%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.051 s |-| |Agent|iast|1.175 s |124.257 ms (11.8%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.184 s |132.832 ms (12.6%)| |Agent|iast_TELEMETRY_OFF|1.172 s |121.524 ms (11.6%)| |Total|tracing|8.498 s |-| |Total|iast|8.978 s |480.415 ms (5.7%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.959 s |460.82 ms (5.4%)| |Total|iast_TELEMETRY_OFF|8.986 s |488.031 ms (5.7%)| ```mermaid gantt title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~942b1e655b, baseline=1.40.0-SNAPSHOT~c4dfa36730 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (670.96 ms) : 0, 670960 BytebuddyAgent [candidate] (670.363 ms) : 0, 670363 GlobalTracer [baseline] (307.271 ms) : 0, 307271 GlobalTracer [candidate] (307.334 ms) : 0, 307334 AppSec [baseline] (51.166 ms) : 0, 51166 AppSec [candidate] (51.484 ms) : 0, 51484 Remote Config [baseline] (685.594 µs) : 0, 686 Remote Config [candidate] (701.521 µs) : 0, 702 Telemetry [baseline] (7.469 ms) : 0, 7469 Telemetry [candidate] (7.478 ms) : 0, 7478 section iast BytebuddyAgent [baseline] (788.132 ms) : 0, 788132 BytebuddyAgent [candidate] (782.43 ms) : 0, 782430 GlobalTracer [baseline] (297.922 ms) : 0, 297922 GlobalTracer [candidate] (295.821 ms) : 0, 295821 AppSec [baseline] (52.936 ms) : 0, 52936 AppSec [candidate] (52.88 ms) : 0, 52880 Remote Config [baseline] (680.874 µs) : 0, 681 Remote Config [candidate] (571.862 µs) : 0, 572 Telemetry [baseline] (8.178 ms) : 0, 8178 Telemetry [candidate] (7.265 ms) : 0, 7265 IAST [baseline] (22.793 ms) : 0, 22793 IAST [candidate] (22.612 ms) : 0, 22612 section iast_HARDCODED_SECRET_DISABLED BytebuddyAgent [baseline] (782.533 ms) : 0, 782533 BytebuddyAgent [candidate] (787.212 ms) : 0, 787212 GlobalTracer [baseline] (296.384 ms) : 0, 296384 GlobalTracer [candidate] (298.507 ms) : 0, 298507 AppSec [baseline] (51.908 ms) : 0, 51908 AppSec [candidate] (53.449 ms) : 0, 53449 Remote Config [baseline] (616.681 µs) : 0, 617 Remote Config [candidate] (601.297 µs) : 0, 601 Telemetry [baseline] (8.938 ms) : 0, 8938 Telemetry [candidate] (7.391 ms) : 0, 7391 IAST [baseline] (21.187 ms) : 0, 21187 IAST [candidate] (22.888 ms) : 0, 22888 section iast_TELEMETRY_OFF BytebuddyAgent [baseline] (783.436 ms) : 0, 783436 BytebuddyAgent [candidate] (778.792 ms) : 0, 778792 GlobalTracer [baseline] (297.52 ms) : 0, 297520 GlobalTracer [candidate] (296.279 ms) : 0, 296279 AppSec [baseline] (52.559 ms) : 0, 52559 AppSec [candidate] (53.474 ms) : 0, 53474 Remote Config [baseline] (598.615 µs) : 0, 599 Remote Config [candidate] (585.911 µs) : 0, 586 Telemetry [baseline] (8.023 ms) : 0, 8023 Telemetry [candidate] (7.36 ms) : 0, 7360 IAST [baseline] (21.976 ms) : 0, 21976 IAST [candidate] (22.352 ms) : 0, 22352 ```
Startup time reports for petclinic ```mermaid gantt title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~942b1e655b, baseline=1.40.0-SNAPSHOT~c4dfa36730 dateFormat X axisFormat %s section tracing Agent [baseline] (1.05 s) : 0, 1049757 Total [baseline] (10.343 s) : 0, 10343414 Agent [candidate] (1.055 s) : 0, 1054543 Total [candidate] (10.434 s) : 0, 10433549 section appsec Agent [baseline] (1.181 s) : 0, 1180789 Total [baseline] (10.63 s) : 0, 10629577 Agent [candidate] (1.179 s) : 0, 1179333 Total [candidate] (10.581 s) : 0, 10580525 section iast Agent [baseline] (1.194 s) : 0, 1193680 Total [baseline] (10.815 s) : 0, 10815072 Agent [candidate] (1.187 s) : 0, 1186779 Total [candidate] (10.877 s) : 0, 10877343 section profiling Agent [baseline] (1.257 s) : 0, 1257187 Total [baseline] (10.7 s) : 0, 10699722 Agent [candidate] (1.252 s) : 0, 1251523 Total [candidate] (10.572 s) : 0, 10571691 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.05 s |-| |Agent|appsec|1.181 s |131.031 ms (12.5%)| |Agent|iast|1.194 s |143.923 ms (13.7%)| |Agent|profiling|1.257 s |207.43 ms (19.8%)| |Total|tracing|10.343 s |-| |Total|appsec|10.63 s |286.163 ms (2.8%)| |Total|iast|10.815 s |471.658 ms (4.6%)| |Total|profiling|10.7 s |356.308 ms (3.4%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.055 s |-| |Agent|appsec|1.179 s |124.79 ms (11.8%)| |Agent|iast|1.187 s |132.235 ms (12.5%)| |Agent|profiling|1.252 s |196.979 ms (18.7%)| |Total|tracing|10.434 s |-| |Total|appsec|10.581 s |146.976 ms (1.4%)| |Total|iast|10.877 s |443.794 ms (4.3%)| |Total|profiling|10.572 s |138.142 ms (1.3%)| ```mermaid gantt title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~942b1e655b, baseline=1.40.0-SNAPSHOT~c4dfa36730 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (670.049 ms) : 0, 670049 BytebuddyAgent [candidate] (672.478 ms) : 0, 672478 GlobalTracer [baseline] (306.793 ms) : 0, 306793 GlobalTracer [candidate] (308.847 ms) : 0, 308847 AppSec [baseline] (51.212 ms) : 0, 51212 AppSec [candidate] (51.451 ms) : 0, 51451 Remote Config [baseline] (691.116 µs) : 0, 691 Remote Config [candidate] (689.717 µs) : 0, 690 Telemetry [baseline] (7.432 ms) : 0, 7432 Telemetry [candidate] (7.468 ms) : 0, 7468 section appsec BytebuddyAgent [baseline] (689.293 ms) : 0, 689293 BytebuddyAgent [candidate] (689.193 ms) : 0, 689193 GlobalTracer [baseline] (300.02 ms) : 0, 300020 GlobalTracer [candidate] (299.833 ms) : 0, 299833 AppSec [baseline] (158.083 ms) : 0, 158083 AppSec [candidate] (158.313 ms) : 0, 158313 Remote Config [baseline] (631.04 µs) : 0, 631 Remote Config [candidate] (632.882 µs) : 0, 633 Telemetry [baseline] (8.258 ms) : 0, 8258 Telemetry [candidate] (7.575 ms) : 0, 7575 IAST [baseline] (21.94 ms) : 0, 21940 IAST [candidate] (20.442 ms) : 0, 20442 section iast BytebuddyAgent [baseline] (795.434 ms) : 0, 795434 BytebuddyAgent [candidate] (788.394 ms) : 0, 788394 GlobalTracer [baseline] (300.34 ms) : 0, 300340 GlobalTracer [candidate] (300.199 ms) : 0, 300199 AppSec [baseline] (53.662 ms) : 0, 53662 AppSec [candidate] (53.456 ms) : 0, 53456 Remote Config [baseline] (583.654 µs) : 0, 584 Remote Config [candidate] (598.166 µs) : 0, 598 Telemetry [baseline] (7.257 ms) : 0, 7257 Telemetry [candidate] (7.449 ms) : 0, 7449 IAST [baseline] (22.622 ms) : 0, 22622 IAST [candidate] (22.945 ms) : 0, 22945 section profiling BytebuddyAgent [baseline] (670.291 ms) : 0, 670291 BytebuddyAgent [candidate] (668.958 ms) : 0, 668958 GlobalTracer [baseline] (391.545 ms) : 0, 391545 GlobalTracer [candidate] (389.151 ms) : 0, 389151 AppSec [baseline] (52.482 ms) : 0, 52482 AppSec [candidate] (51.981 ms) : 0, 51981 Remote Config [baseline] (688.439 µs) : 0, 688 Remote Config [candidate] (683.409 µs) : 0, 683 Telemetry [baseline] (7.439 ms) : 0, 7439 Telemetry [candidate] (7.342 ms) : 0, 7342 ProfilingAgent [baseline] (96.633 ms) : 0, 96633 ProfilingAgent [candidate] (95.343 ms) : 0, 95343 Profiling [baseline] (96.657 ms) : 0, 96657 Profiling [candidate] (95.367 ms) : 0, 95367 ```

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-09T12:29:51 2024-09-09T12:36:37
git_branch master jpbempel/fix-flaky-ibm
git_commit_date 1725881796 1725884395
git_commit_sha c4dfa36730 942b1e655b
release_version 1.40.0-SNAPSHOT~c4dfa36730 1.40.0-SNAPSHOT~942b1e655b
start_time 2024-09-09T12:29:37 2024-09-09T12:36:24
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1725885741 |1725885741 | |ci_job_id |631799209 |631799209 | |ci_pipeline_id|43844844 |43844844 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 19 unstable metrics.

Request duration reports for petclinic ```mermaid gantt title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~942b1e655b, baseline=1.40.0-SNAPSHOT~c4dfa36730 dateFormat X axisFormat %s section baseline no_agent (1.344 ms) : 1324, 1364 . : milestone, 1344, appsec (1.696 ms) : 1672, 1721 . : milestone, 1696, appsec_no_iast (1.729 ms) : 1705, 1753 . : milestone, 1729, iast (1.477 ms) : 1455, 1500 . : milestone, 1477, profiling (1.488 ms) : 1465, 1512 . : milestone, 1488, tracing (1.461 ms) : 1436, 1486 . : milestone, 1461, section candidate no_agent (1.323 ms) : 1304, 1343 . : milestone, 1323, appsec (1.722 ms) : 1698, 1745 . : milestone, 1722, appsec_no_iast (1.711 ms) : 1687, 1736 . : milestone, 1711, iast (1.47 ms) : 1447, 1492 . : milestone, 1470, profiling (1.496 ms) : 1472, 1520 . : milestone, 1496, tracing (1.457 ms) : 1433, 1481 . : milestone, 1457, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.344 ms [1.324 ms, 1.364 ms]|-| |appsec|1.696 ms [1.672 ms, 1.721 ms]|352.256 µs (26.2%)| |appsec_no_iast|1.729 ms [1.705 ms, 1.753 ms]|384.477 µs (28.6%)| |iast|1.477 ms [1.455 ms, 1.5 ms]|132.99 µs (9.9%)| |profiling|1.488 ms [1.465 ms, 1.512 ms]|144.238 µs (10.7%)| |tracing|1.461 ms [1.436 ms, 1.486 ms]|116.806 µs (8.7%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.323 ms [1.304 ms, 1.343 ms]|-| |appsec|1.722 ms [1.698 ms, 1.745 ms]|398.227 µs (30.1%)| |appsec_no_iast|1.711 ms [1.687 ms, 1.736 ms]|387.902 µs (29.3%)| |iast|1.47 ms [1.447 ms, 1.492 ms]|146.169 µs (11.0%)| |profiling|1.496 ms [1.472 ms, 1.52 ms]|173.001 µs (13.1%)| |tracing|1.457 ms [1.433 ms, 1.481 ms]|133.849 µs (10.1%)|
Request duration reports for insecure-bank ```mermaid gantt title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~942b1e655b, baseline=1.40.0-SNAPSHOT~c4dfa36730 dateFormat X axisFormat %s section baseline no_agent (364.676 µs) : 344, 385 . : milestone, 365, iast (479.125 µs) : 456, 502 . : milestone, 479, iast_FULL (548.227 µs) : 527, 569 . : milestone, 548, iast_GLOBAL (505.201 µs) : 484, 527 . : milestone, 505, iast_HARDCODED_SECRET_DISABLED (473.615 µs) : 452, 495 . : milestone, 474, iast_INACTIVE (432.927 µs) : 413, 453 . : milestone, 433, iast_TELEMETRY_OFF (471.776 µs) : 449, 495 . : milestone, 472, tracing (445.847 µs) : 424, 468 . : milestone, 446, section candidate no_agent (363.308 µs) : 343, 383 . : milestone, 363, iast (481.692 µs) : 460, 504 . : milestone, 482, iast_FULL (544.834 µs) : 523, 567 . : milestone, 545, iast_GLOBAL (495.829 µs) : 475, 517 . : milestone, 496, iast_HARDCODED_SECRET_DISABLED (478.003 µs) : 455, 501 . : milestone, 478, iast_INACTIVE (433.844 µs) : 413, 454 . : milestone, 434, iast_TELEMETRY_OFF (468.876 µs) : 446, 492 . : milestone, 469, tracing (430.325 µs) : 410, 450 . : milestone, 430, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|364.676 µs [343.855 µs, 385.496 µs]|-| |iast|479.125 µs [456.477 µs, 501.772 µs]|114.449 µs (31.4%)| |iast_FULL|548.227 µs [527.0 µs, 569.453 µs]|183.551 µs (50.3%)| |iast_GLOBAL|505.201 µs [483.818 µs, 526.585 µs]|140.526 µs (38.5%)| |iast_HARDCODED_SECRET_DISABLED|473.615 µs [452.284 µs, 494.945 µs]|108.939 µs (29.9%)| |iast_INACTIVE|432.927 µs [412.672 µs, 453.182 µs]|68.252 µs (18.7%)| |iast_TELEMETRY_OFF|471.776 µs [448.884 µs, 494.669 µs]|107.101 µs (29.4%)| |tracing|445.847 µs [424.011 µs, 467.684 µs]|81.172 µs (22.3%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|363.308 µs [343.406 µs, 383.21 µs]|-| |iast|481.692 µs [459.808 µs, 503.575 µs]|118.383 µs (32.6%)| |iast_FULL|544.834 µs [523.066 µs, 566.601 µs]|181.525 µs (50.0%)| |iast_GLOBAL|495.829 µs [474.619 µs, 517.039 µs]|132.52 µs (36.5%)| |iast_HARDCODED_SECRET_DISABLED|478.003 µs [455.478 µs, 500.528 µs]|114.695 µs (31.6%)| |iast_INACTIVE|433.844 µs [413.43 µs, 454.258 µs]|70.535 µs (19.4%)| |iast_TELEMETRY_OFF|468.876 µs [445.984 µs, 491.768 µs]|105.568 µs (29.1%)| |tracing|430.325 µs [410.364 µs, 450.287 µs]|67.017 µs (18.4%)|

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-ibm
git_commit_date 1725884741 1725884395
git_commit_sha 5ddb19db3a 942b1e655b
release_version 1.40.0-SNAPSHOT~5ddb19db3a 1.40.0-SNAPSHOT~942b1e655b
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |biojava |biojava | |ci_job_date |1725886464 |1725886464 | |ci_job_id |631799210 |631799210 | |ci_pipeline_id|43844844 |43844844 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |appsec |appsec |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava ```mermaid gantt title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~942b1e655b, baseline=1.40.0-SNAPSHOT~5ddb19db3a dateFormat X axisFormat %s section baseline no_agent (15.027 s) : 15027000, 15027000 . : milestone, 15027000, appsec (14.988 s) : 14988000, 14988000 . : milestone, 14988000, iast (18.623 s) : 18623000, 18623000 . : milestone, 18623000, iast_GLOBAL (18.237 s) : 18237000, 18237000 . : milestone, 18237000, profiling (15.781 s) : 15781000, 15781000 . : milestone, 15781000, tracing (15.131 s) : 15131000, 15131000 . : milestone, 15131000, section candidate no_agent (15.144 s) : 15144000, 15144000 . : milestone, 15144000, appsec (15.148 s) : 15148000, 15148000 . : milestone, 15148000, iast (18.901 s) : 18901000, 18901000 . : milestone, 18901000, iast_GLOBAL (17.95 s) : 17950000, 17950000 . : milestone, 17950000, profiling (15.41 s) : 15410000, 15410000 . : milestone, 15410000, tracing (15.09 s) : 15090000, 15090000 . : milestone, 15090000, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.027 s [15.027 s, 15.027 s]|-| |appsec|14.988 s [14.988 s, 14.988 s]|-39.0 ms (-0.3%)| |iast|18.623 s [18.623 s, 18.623 s]|3.596 s (23.9%)| |iast_GLOBAL|18.237 s [18.237 s, 18.237 s]|3.21 s (21.4%)| |profiling|15.781 s [15.781 s, 15.781 s]|754.0 ms (5.0%)| |tracing|15.131 s [15.131 s, 15.131 s]|104.0 ms (0.7%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.144 s [15.144 s, 15.144 s]|-| |appsec|15.148 s [15.148 s, 15.148 s]|4.0 ms (0.0%)| |iast|18.901 s [18.901 s, 18.901 s]|3.757 s (24.8%)| |iast_GLOBAL|17.95 s [17.95 s, 17.95 s]|2.806 s (18.5%)| |profiling|15.41 s [15.41 s, 15.41 s]|266.0 ms (1.8%)| |tracing|15.09 s [15.09 s, 15.09 s]|-54.0 ms (-0.4%)|
Execution time for tomcat ```mermaid gantt title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~942b1e655b, baseline=1.40.0-SNAPSHOT~5ddb19db3a dateFormat X axisFormat %s section baseline no_agent (1.453 ms) : 1442, 1465 . : milestone, 1453, appsec (2.211 ms) : 2176, 2245 . : milestone, 2211, iast (1.977 ms) : 1934, 2020 . : milestone, 1977, iast_GLOBAL (2.011 ms) : 1968, 2054 . : milestone, 2011, profiling (1.858 ms) : 1822, 1893 . : milestone, 1858, tracing (1.838 ms) : 1805, 1872 . : milestone, 1838, section candidate no_agent (1.451 ms) : 1440, 1463 . : milestone, 1451, appsec (2.215 ms) : 2180, 2250 . : milestone, 2215, iast (1.966 ms) : 1924, 2008 . : milestone, 1966, iast_GLOBAL (2.015 ms) : 1972, 2058 . : milestone, 2015, profiling (1.857 ms) : 1823, 1891 . : milestone, 1857, tracing (1.84 ms) : 1807, 1874 . : milestone, 1840, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.453 ms [1.442 ms, 1.465 ms]|-| |appsec|2.211 ms [2.176 ms, 2.245 ms]|757.478 µs (52.1%)| |iast|1.977 ms [1.934 ms, 2.02 ms]|523.275 µs (36.0%)| |iast_GLOBAL|2.011 ms [1.968 ms, 2.054 ms]|557.708 µs (38.4%)| |profiling|1.858 ms [1.822 ms, 1.893 ms]|404.512 µs (27.8%)| |tracing|1.838 ms [1.805 ms, 1.872 ms]|385.089 µs (26.5%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.451 ms [1.44 ms, 1.463 ms]|-| |appsec|2.215 ms [2.18 ms, 2.25 ms]|763.764 µs (52.6%)| |iast|1.966 ms [1.924 ms, 2.008 ms]|514.973 µs (35.5%)| |iast_GLOBAL|2.015 ms [1.972 ms, 2.058 ms]|563.472 µs (38.8%)| |profiling|1.857 ms [1.823 ms, 1.891 ms]|405.852 µs (28.0%)| |tracing|1.84 ms [1.807 ms, 1.874 ms]|389.074 µs (26.8%)|