search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
588 stars 290 forks source link

Fix automatic coverage includes calculation for headless test sessions #7804

Closed nikita-tkachenko-datadog closed 1 month ago

nikita-tkachenko-datadog commented 1 month ago

Duplicates #7784, merged by mistake

pr-commenter[bot] commented 1 month ago

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/backport-pr-7784
git_commit_date 1729245368 1729254359
git_commit_sha 52cdab0c9c c766655939
release_version 1.42.0-SNAPSHOT~52cdab0c9c 1.42.0-SNAPSHOT~c766655939
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1729256809 |1729256809 | |ci_job_id |676764400 |676764400 | |ci_pipeline_id|46908678 |46908678 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |module |Agent |Agent | |parent |None |None | |variant |iast |iast |

Summary

Found 1 performance improvements and 2 performance regressions! Performance is the same for 51 metrics, 9 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:AppSec better
[-6.275ms; -1.279ms] or [-10.836%; -2.209%]		 54.130ms 57.907ms
scenario:startup:petclinic:profiling:ProfilingAgent worse
[+4.966ms; +11.055ms] or [+5.512%; +12.269%]		 98.111ms 90.100ms
scenario:startup:petclinic:profiling:Profiling worse
[+4.966ms; +11.055ms] or [+5.510%; +12.266%]		 98.134ms 90.124ms
Startup time reports for insecure-bank ```mermaid gantt title insecure-bank - global startup overhead: candidate=1.42.0-SNAPSHOT~c766655939, baseline=1.42.0-SNAPSHOT~52cdab0c9c dateFormat X axisFormat %s section tracing Agent [baseline] (1.081 s) : 0, 1080911 Total [baseline] (8.628 s) : 0, 8628089 Agent [candidate] (1.071 s) : 0, 1070514 Total [candidate] (8.568 s) : 0, 8568205 section iast Agent [baseline] (1.199 s) : 0, 1199305 Total [baseline] (9.071 s) : 0, 9071425 Agent [candidate] (1.205 s) : 0, 1204912 Total [candidate] (9.169 s) : 0, 9169422 section iast_HARDCODED_SECRET_DISABLED Agent [baseline] (1.201 s) : 0, 1200516 Total [baseline] (9.034 s) : 0, 9033889 Agent [candidate] (1.214 s) : 0, 1214426 Total [candidate] (9.111 s) : 0, 9111384 section iast_TELEMETRY_OFF Agent [baseline] (1.198 s) : 0, 1197843 Total [baseline] (9.095 s) : 0, 9095222 Agent [candidate] (1.196 s) : 0, 1195772 Total [candidate] (9.056 s) : 0, 9055811 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.081 s |-| |Agent|iast|1.199 s |118.394 ms (11.0%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.201 s |119.605 ms (11.1%)| |Agent|iast_TELEMETRY_OFF|1.198 s |116.932 ms (10.8%)| |Total|tracing|8.628 s |-| |Total|iast|9.071 s |443.336 ms (5.1%)| |Total|iast_HARDCODED_SECRET_DISABLED|9.034 s |405.8 ms (4.7%)| |Total|iast_TELEMETRY_OFF|9.095 s |467.133 ms (5.4%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.071 s |-| |Agent|iast|1.205 s |134.397 ms (12.6%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.214 s |143.912 ms (13.4%)| |Agent|iast_TELEMETRY_OFF|1.196 s |125.258 ms (11.7%)| |Total|tracing|8.568 s |-| |Total|iast|9.169 s |601.217 ms (7.0%)| |Total|iast_HARDCODED_SECRET_DISABLED|9.111 s |543.179 ms (6.3%)| |Total|iast_TELEMETRY_OFF|9.056 s |487.606 ms (5.7%)| ```mermaid gantt title insecure-bank - break down per module: candidate=1.42.0-SNAPSHOT~c766655939, baseline=1.42.0-SNAPSHOT~52cdab0c9c dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (687.385 ms) : 0, 687385 BytebuddyAgent [candidate] (682.635 ms) : 0, 682635 GlobalTracer [baseline] (314.381 ms) : 0, 314381 GlobalTracer [candidate] (312.15 ms) : 0, 312150 AppSec [baseline] (54.54 ms) : 0, 54540 AppSec [candidate] (53.988 ms) : 0, 53988 Remote Config [baseline] (673.814 µs) : 0, 674 Remote Config [candidate] (662.863 µs) : 0, 663 Telemetry [baseline] (10.27 ms) : 0, 10270 Telemetry [candidate] (7.418 ms) : 0, 7418 section iast BytebuddyAgent [baseline] (799.291 ms) : 0, 799291 BytebuddyAgent [candidate] (802.566 ms) : 0, 802566 GlobalTracer [baseline] (300.985 ms) : 0, 300985 GlobalTracer [candidate] (302.624 ms) : 0, 302624 AppSec [baseline] (57.249 ms) : 0, 57249 AppSec [candidate] (56.203 ms) : 0, 56203 IAST [baseline] (20.485 ms) : 0, 20485 IAST [candidate] (22.128 ms) : 0, 22128 Remote Config [baseline] (620.301 µs) : 0, 620 Remote Config [candidate] (605.991 µs) : 0, 606 Telemetry [baseline] (7.057 ms) : 0, 7057 Telemetry [candidate] (7.075 ms) : 0, 7075 section iast_HARDCODED_SECRET_DISABLED BytebuddyAgent [baseline] (800.097 ms) : 0, 800097 BytebuddyAgent [candidate] (808.747 ms) : 0, 808747 GlobalTracer [baseline] (300.862 ms) : 0, 300862 GlobalTracer [candidate] (305.015 ms) : 0, 305015 AppSec [baseline] (57.299 ms) : 0, 57299 AppSec [candidate] (58.421 ms) : 0, 58421 IAST [baseline] (21.065 ms) : 0, 21065 IAST [candidate] (20.601 ms) : 0, 20601 Remote Config [baseline] (594.759 µs) : 0, 595 Remote Config [candidate] (613.371 µs) : 0, 613 Telemetry [baseline] (7.039 ms) : 0, 7039 Telemetry [candidate] (7.191 ms) : 0, 7191 section iast_TELEMETRY_OFF BytebuddyAgent [baseline] (797.448 ms) : 0, 797448 BytebuddyAgent [candidate] (795.107 ms) : 0, 795107 GlobalTracer [baseline] (301.291 ms) : 0, 301291 GlobalTracer [candidate] (301.506 ms) : 0, 301506 AppSec [baseline] (57.907 ms) : 0, 57907 AppSec [candidate] (54.13 ms) : 0, 54130 IAST [baseline] (20.007 ms) : 0, 20007 IAST [candidate] (23.954 ms) : 0, 23954 Remote Config [baseline] (603.159 µs) : 0, 603 Remote Config [candidate] (588.505 µs) : 0, 589 Telemetry [baseline] (7.001 ms) : 0, 7001 Telemetry [candidate] (6.838 ms) : 0, 6838 ```
Startup time reports for petclinic ```mermaid gantt title petclinic - global startup overhead: candidate=1.42.0-SNAPSHOT~c766655939, baseline=1.42.0-SNAPSHOT~52cdab0c9c dateFormat X axisFormat %s section tracing Agent [baseline] (1.074 s) : 0, 1073640 Total [baseline] (10.423 s) : 0, 10422841 Agent [candidate] (1.07 s) : 0, 1069725 Total [candidate] (10.4 s) : 0, 10400478 section appsec Agent [baseline] (1.205 s) : 0, 1205398 Total [baseline] (10.548 s) : 0, 10548070 Agent [candidate] (1.212 s) : 0, 1211745 Total [candidate] (10.63 s) : 0, 10629920 section iast Agent [baseline] (1.198 s) : 0, 1197814 Total [baseline] (10.87 s) : 0, 10869713 Agent [candidate] (1.209 s) : 0, 1208671 Total [candidate] (10.915 s) : 0, 10915099 section profiling Agent [baseline] (1.275 s) : 0, 1274972 Total [baseline] (10.693 s) : 0, 10693397 Agent [candidate] (1.285 s) : 0, 1284846 Total [candidate] (10.745 s) : 0, 10744698 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.074 s |-| |Agent|appsec|1.205 s |131.758 ms (12.3%)| |Agent|iast|1.198 s |124.174 ms (11.6%)| |Agent|profiling|1.275 s |201.332 ms (18.8%)| |Total|tracing|10.423 s |-| |Total|appsec|10.548 s |125.23 ms (1.2%)| |Total|iast|10.87 s |446.872 ms (4.3%)| |Total|profiling|10.693 s |270.556 ms (2.6%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.07 s |-| |Agent|appsec|1.212 s |142.02 ms (13.3%)| |Agent|iast|1.209 s |138.946 ms (13.0%)| |Agent|profiling|1.285 s |215.121 ms (20.1%)| |Total|tracing|10.4 s |-| |Total|appsec|10.63 s |229.442 ms (2.2%)| |Total|iast|10.915 s |514.621 ms (4.9%)| |Total|profiling|10.745 s |344.22 ms (3.3%)| ```mermaid gantt title petclinic - break down per module: candidate=1.42.0-SNAPSHOT~c766655939, baseline=1.42.0-SNAPSHOT~52cdab0c9c dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (684.39 ms) : 0, 684390 BytebuddyAgent [candidate] (682.255 ms) : 0, 682255 GlobalTracer [baseline] (311.663 ms) : 0, 311663 GlobalTracer [candidate] (311.795 ms) : 0, 311795 AppSec [baseline] (53.72 ms) : 0, 53720 AppSec [candidate] (53.919 ms) : 0, 53919 Remote Config [baseline] (669.281 µs) : 0, 669 Remote Config [candidate] (674.316 µs) : 0, 674 Telemetry [baseline] (9.624 ms) : 0, 9624 Telemetry [candidate] (7.452 ms) : 0, 7452 section appsec BytebuddyAgent [baseline] (700.174 ms) : 0, 700174 BytebuddyAgent [candidate] (704.491 ms) : 0, 704491 GlobalTracer [baseline] (308.793 ms) : 0, 308793 GlobalTracer [candidate] (310.93 ms) : 0, 310930 AppSec [baseline] (165.664 ms) : 0, 165664 AppSec [candidate] (162.034 ms) : 0, 162034 Remote Config [baseline] (629.914 µs) : 0, 630 Remote Config [candidate] (632.678 µs) : 0, 633 Telemetry [baseline] (7.059 ms) : 0, 7059 Telemetry [candidate] (9.188 ms) : 0, 9188 IAST [baseline] (19.033 ms) : 0, 19033 IAST [candidate] (21.548 ms) : 0, 21548 section iast BytebuddyAgent [baseline] (798.115 ms) : 0, 798115 BytebuddyAgent [candidate] (803.67 ms) : 0, 803670 GlobalTracer [baseline] (300.66 ms) : 0, 300660 GlobalTracer [candidate] (304.32 ms) : 0, 304320 AppSec [baseline] (57.455 ms) : 0, 57455 AppSec [candidate] (58.207 ms) : 0, 58207 Remote Config [baseline] (603.856 µs) : 0, 604 Remote Config [candidate] (623.07 µs) : 0, 623 Telemetry [baseline] (7.012 ms) : 0, 7012 Telemetry [candidate] (7.272 ms) : 0, 7272 IAST [baseline] (20.395 ms) : 0, 20395 IAST [candidate] (20.863 ms) : 0, 20863 section profiling ProfilingAgent [baseline] (90.1 ms) : 0, 90100 ProfilingAgent [candidate] (98.111 ms) : 0, 98111 BytebuddyAgent [baseline] (681.38 ms) : 0, 681380 BytebuddyAgent [candidate] (684.895 ms) : 0, 684895 GlobalTracer [baseline] (395.958 ms) : 0, 395958 GlobalTracer [candidate] (399.39 ms) : 0, 399390 AppSec [baseline] (54.489 ms) : 0, 54489 AppSec [candidate] (55.124 ms) : 0, 55124 Remote Config [baseline] (648.581 µs) : 0, 649 Remote Config [candidate] (659.475 µs) : 0, 659 Telemetry [baseline] (13.643 ms) : 0, 13643 Telemetry [candidate] (7.586 ms) : 0, 7586 Profiling [baseline] (90.124 ms) : 0, 90124 Profiling [candidate] (98.134 ms) : 0, 98134 ```

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-10-18T12:37:18 2024-10-18T12:44:09
git_branch master nikita-tkachenko/backport-pr-7784
git_commit_date 1729245368 1729254359
git_commit_sha 52cdab0c9c c766655939
release_version 1.42.0-SNAPSHOT~52cdab0c9c 1.42.0-SNAPSHOT~c766655939
start_time 2024-10-18T12:37:05 2024-10-18T12:43:56
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1729255795 |1729255795 | |ci_job_id |676764401 |676764401 | |ci_pipeline_id|46908678 |46908678 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for insecure-bank ```mermaid gantt title insecure-bank - request duration [CI 0.99] : candidate=1.42.0-SNAPSHOT~c766655939, baseline=1.42.0-SNAPSHOT~52cdab0c9c dateFormat X axisFormat %s section baseline no_agent (374.646 µs) : 355, 394 . : milestone, 375, iast (480.082 µs) : 459, 501 . : milestone, 480, iast_FULL (551.885 µs) : 530, 573 . : milestone, 552, iast_GLOBAL (525.246 µs) : 502, 549 . : milestone, 525, iast_HARDCODED_SECRET_DISABLED (479.804 µs) : 458, 501 . : milestone, 480, iast_INACTIVE (446.055 µs) : 425, 467 . : milestone, 446, iast_TELEMETRY_OFF (475.987 µs) : 455, 497 . : milestone, 476, tracing (443.116 µs) : 423, 464 . : milestone, 443, section candidate no_agent (370.015 µs) : 351, 389 . : milestone, 370, iast (484.105 µs) : 463, 505 . : milestone, 484, iast_FULL (554.156 µs) : 533, 576 . : milestone, 554, iast_GLOBAL (502.195 µs) : 481, 523 . : milestone, 502, iast_HARDCODED_SECRET_DISABLED (485.756 µs) : 464, 507 . : milestone, 486, iast_INACTIVE (447.575 µs) : 427, 468 . : milestone, 448, iast_TELEMETRY_OFF (473.484 µs) : 452, 494 . : milestone, 473, tracing (442.706 µs) : 422, 463 . : milestone, 443, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|374.646 µs [355.011 µs, 394.281 µs]|-| |iast|480.082 µs [458.975 µs, 501.189 µs]|105.436 µs (28.1%)| |iast_FULL|551.885 µs [530.407 µs, 573.362 µs]|177.238 µs (47.3%)| |iast_GLOBAL|525.246 µs [501.721 µs, 548.77 µs]|150.6 µs (40.2%)| |iast_HARDCODED_SECRET_DISABLED|479.804 µs [458.463 µs, 501.144 µs]|105.158 µs (28.1%)| |iast_INACTIVE|446.055 µs [425.395 µs, 466.715 µs]|71.409 µs (19.1%)| |iast_TELEMETRY_OFF|475.987 µs [454.56 µs, 497.414 µs]|101.341 µs (27.0%)| |tracing|443.116 µs [422.586 µs, 463.646 µs]|68.47 µs (18.3%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|370.015 µs [350.662 µs, 389.367 µs]|-| |iast|484.105 µs [462.762 µs, 505.449 µs]|114.091 µs (30.8%)| |iast_FULL|554.156 µs [532.7 µs, 575.612 µs]|184.141 µs (49.8%)| |iast_GLOBAL|502.195 µs [481.095 µs, 523.295 µs]|132.18 µs (35.7%)| |iast_HARDCODED_SECRET_DISABLED|485.756 µs [464.112 µs, 507.399 µs]|115.741 µs (31.3%)| |iast_INACTIVE|447.575 µs [426.866 µs, 468.283 µs]|77.56 µs (21.0%)| |iast_TELEMETRY_OFF|473.484 µs [452.47 µs, 494.498 µs]|103.47 µs (28.0%)| |tracing|442.706 µs [422.225 µs, 463.188 µs]|72.692 µs (19.6%)|
Request duration reports for petclinic ```mermaid gantt title petclinic - request duration [CI 0.99] : candidate=1.42.0-SNAPSHOT~c766655939, baseline=1.42.0-SNAPSHOT~52cdab0c9c dateFormat X axisFormat %s section baseline no_agent (1.325 ms) : 1306, 1345 . : milestone, 1325, appsec (1.718 ms) : 1694, 1743 . : milestone, 1718, appsec_no_iast (1.722 ms) : 1697, 1746 . : milestone, 1722, iast (1.464 ms) : 1441, 1487 . : milestone, 1464, profiling (1.495 ms) : 1472, 1518 . : milestone, 1495, tracing (1.474 ms) : 1451, 1497 . : milestone, 1474, section candidate no_agent (1.364 ms) : 1345, 1384 . : milestone, 1364, appsec (1.719 ms) : 1695, 1744 . : milestone, 1719, appsec_no_iast (1.721 ms) : 1697, 1745 . : milestone, 1721, iast (1.473 ms) : 1450, 1496 . : milestone, 1473, profiling (1.468 ms) : 1444, 1492 . : milestone, 1468, tracing (1.478 ms) : 1454, 1502 . : milestone, 1478, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.325 ms [1.306 ms, 1.345 ms]|-| |appsec|1.718 ms [1.694 ms, 1.743 ms]|392.765 µs (29.6%)| |appsec_no_iast|1.722 ms [1.697 ms, 1.746 ms]|396.282 µs (29.9%)| |iast|1.464 ms [1.441 ms, 1.487 ms]|138.85 µs (10.5%)| |profiling|1.495 ms [1.472 ms, 1.518 ms]|169.625 µs (12.8%)| |tracing|1.474 ms [1.451 ms, 1.497 ms]|148.872 µs (11.2%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.364 ms [1.345 ms, 1.384 ms]|-| |appsec|1.719 ms [1.695 ms, 1.744 ms]|355.244 µs (26.0%)| |appsec_no_iast|1.721 ms [1.697 ms, 1.745 ms]|357.266 µs (26.2%)| |iast|1.473 ms [1.45 ms, 1.496 ms]|109.116 µs (8.0%)| |profiling|1.468 ms [1.444 ms, 1.492 ms]|103.859 µs (7.6%)| |tracing|1.478 ms [1.454 ms, 1.502 ms]|113.722 µs (8.3%)|

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/backport-pr-7784
git_commit_date 1729245368 1729254359
git_commit_sha 52cdab0c9c c766655939
release_version 1.42.0-SNAPSHOT~52cdab0c9c 1.42.0-SNAPSHOT~c766655939
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |biojava |biojava | |ci_job_date |1729256335 |1729256335 | |ci_job_id |676764402 |676764402 | |ci_pipeline_id|46908678 |46908678 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |appsec |appsec |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat ```mermaid gantt title tomcat - execution time [CI 0.99] : candidate=1.42.0-SNAPSHOT~c766655939, baseline=1.42.0-SNAPSHOT~52cdab0c9c dateFormat X axisFormat %s section baseline no_agent (1.459 ms) : 1447, 1470 . : milestone, 1459, appsec (2.309 ms) : 2268, 2350 . : milestone, 2309, iast (2.07 ms) : 2018, 2122 . : milestone, 2070, iast_GLOBAL (2.097 ms) : 2046, 2149 . : milestone, 2097, profiling (1.934 ms) : 1892, 1976 . : milestone, 1934, tracing (1.907 ms) : 1867, 1946 . : milestone, 1907, section candidate no_agent (1.458 ms) : 1447, 1470 . : milestone, 1458, appsec (2.306 ms) : 2265, 2347 . : milestone, 2306, iast (2.066 ms) : 2014, 2118 . : milestone, 2066, iast_GLOBAL (2.094 ms) : 2042, 2145 . : milestone, 2094, profiling (1.944 ms) : 1900, 1987 . : milestone, 1944, tracing (1.9 ms) : 1860, 1939 . : milestone, 1900, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.459 ms [1.447 ms, 1.47 ms]|-| |appsec|2.309 ms [2.268 ms, 2.35 ms]|850.002 µs (58.3%)| |iast|2.07 ms [2.018 ms, 2.122 ms]|611.187 µs (41.9%)| |iast_GLOBAL|2.097 ms [2.046 ms, 2.149 ms]|638.696 µs (43.8%)| |profiling|1.934 ms [1.892 ms, 1.976 ms]|475.414 µs (32.6%)| |tracing|1.907 ms [1.867 ms, 1.946 ms]|447.907 µs (30.7%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.458 ms [1.447 ms, 1.47 ms]|-| |appsec|2.306 ms [2.265 ms, 2.347 ms]|847.539 µs (58.1%)| |iast|2.066 ms [2.014 ms, 2.118 ms]|608.099 µs (41.7%)| |iast_GLOBAL|2.094 ms [2.042 ms, 2.145 ms]|635.562 µs (43.6%)| |profiling|1.944 ms [1.9 ms, 1.987 ms]|485.555 µs (33.3%)| |tracing|1.9 ms [1.86 ms, 1.939 ms]|441.478 µs (30.3%)|
Execution time for biojava ```mermaid gantt title biojava - execution time [CI 0.99] : candidate=1.42.0-SNAPSHOT~c766655939, baseline=1.42.0-SNAPSHOT~52cdab0c9c dateFormat X axisFormat %s section baseline no_agent (15.63 s) : 15630000, 15630000 . : milestone, 15630000, appsec (15.222 s) : 15222000, 15222000 . : milestone, 15222000, iast (18.812 s) : 18812000, 18812000 . : milestone, 18812000, iast_GLOBAL (18.336 s) : 18336000, 18336000 . : milestone, 18336000, profiling (15.163 s) : 15163000, 15163000 . : milestone, 15163000, tracing (15.46 s) : 15460000, 15460000 . : milestone, 15460000, section candidate no_agent (15.449 s) : 15449000, 15449000 . : milestone, 15449000, appsec (15.338 s) : 15338000, 15338000 . : milestone, 15338000, iast (18.973 s) : 18973000, 18973000 . : milestone, 18973000, iast_GLOBAL (18.106 s) : 18106000, 18106000 . : milestone, 18106000, profiling (15.269 s) : 15269000, 15269000 . : milestone, 15269000, tracing (15.304 s) : 15304000, 15304000 . : milestone, 15304000, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.63 s [15.63 s, 15.63 s]|-| |appsec|15.222 s [15.222 s, 15.222 s]|-408.0 ms (-2.6%)| |iast|18.812 s [18.812 s, 18.812 s]|3.182 s (20.4%)| |iast_GLOBAL|18.336 s [18.336 s, 18.336 s]|2.706 s (17.3%)| |profiling|15.163 s [15.163 s, 15.163 s]|-467.0 ms (-3.0%)| |tracing|15.46 s [15.46 s, 15.46 s]|-170.0 ms (-1.1%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.449 s [15.449 s, 15.449 s]|-| |appsec|15.338 s [15.338 s, 15.338 s]|-111.0 ms (-0.7%)| |iast|18.973 s [18.973 s, 18.973 s]|3.524 s (22.8%)| |iast_GLOBAL|18.106 s [18.106 s, 18.106 s]|2.657 s (17.2%)| |profiling|15.269 s [15.269 s, 15.269 s]|-180.0 ms (-1.2%)| |tracing|15.304 s [15.304 s, 15.304 s]|-145.0 ms (-0.9%)|