search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
572 stars 284 forks source link

Fix flaky test on TomcatServlet3TestAsync #7550

Closed smola closed 2 weeks ago

smola commented 3 weeks ago

What Does This Do

Under high load, the order of spans by duration might be altered. Make sure we check local root spans, and that we allow for different order in traces and access logs.

Motivation

Additional Notes

Contributor Checklist

pr-commenter[bot] commented 3 weeks ago

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/tomcat-async-flaky
git_commit_date 1726062980 1726064446
git_commit_sha 4abf841368 4f0abe12a2
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~4f0abe12a2
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1726066757 |1726066757 | |ci_job_id |635568523 |635568523 | |ci_pipeline_id|44081872 |44081872 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |module |Agent |Agent | |parent |None |None | |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 12 unstable metrics.

Startup time reports for petclinic ```mermaid gantt title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368 dateFormat X axisFormat %s section tracing Agent [baseline] (1.07 s) : 0, 1069702 Total [baseline] (10.333 s) : 0, 10333364 Agent [candidate] (1.071 s) : 0, 1070851 Total [candidate] (10.383 s) : 0, 10382745 section appsec Agent [baseline] (1.194 s) : 0, 1194163 Total [baseline] (10.581 s) : 0, 10581449 Agent [candidate] (1.201 s) : 0, 1201266 Total [candidate] (10.596 s) : 0, 10595997 section iast Agent [baseline] (1.189 s) : 0, 1189280 Total [baseline] (10.798 s) : 0, 10798165 Agent [candidate] (1.191 s) : 0, 1191377 Total [candidate] (10.838 s) : 0, 10838252 section profiling Agent [baseline] (1.262 s) : 0, 1261836 Total [baseline] (10.642 s) : 0, 10642303 Agent [candidate] (1.27 s) : 0, 1270035 Total [candidate] (10.587 s) : 0, 10586594 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.07 s |-| |Agent|appsec|1.194 s |124.461 ms (11.6%)| |Agent|iast|1.189 s |119.578 ms (11.2%)| |Agent|profiling|1.262 s |192.134 ms (18.0%)| |Total|tracing|10.333 s |-| |Total|appsec|10.581 s |248.085 ms (2.4%)| |Total|iast|10.798 s |464.801 ms (4.5%)| |Total|profiling|10.642 s |308.94 ms (3.0%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.071 s |-| |Agent|appsec|1.201 s |130.416 ms (12.2%)| |Agent|iast|1.191 s |120.526 ms (11.3%)| |Agent|profiling|1.27 s |199.185 ms (18.6%)| |Total|tracing|10.383 s |-| |Total|appsec|10.596 s |213.252 ms (2.1%)| |Total|iast|10.838 s |455.507 ms (4.4%)| |Total|profiling|10.587 s |203.849 ms (2.0%)| ```mermaid gantt title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (683.112 ms) : 0, 683112 BytebuddyAgent [candidate] (683.388 ms) : 0, 683388 GlobalTracer [baseline] (313.145 ms) : 0, 313145 GlobalTracer [candidate] (313.267 ms) : 0, 313267 AppSec [baseline] (51.565 ms) : 0, 51565 AppSec [candidate] (52.206 ms) : 0, 52206 Remote Config [baseline] (701.256 µs) : 0, 701 Remote Config [candidate] (714.873 µs) : 0, 715 Telemetry [baseline] (7.475 ms) : 0, 7475 Telemetry [candidate] (7.538 ms) : 0, 7538 section appsec BytebuddyAgent [baseline] (697.998 ms) : 0, 697998 BytebuddyAgent [candidate] (701.593 ms) : 0, 701593 GlobalTracer [baseline] (303.654 ms) : 0, 303654 GlobalTracer [candidate] (305.051 ms) : 0, 305051 AppSec [baseline] (160.367 ms) : 0, 160367 AppSec [candidate] (161.15 ms) : 0, 161150 IAST [baseline] (19.807 ms) : 0, 19807 IAST [candidate] (20.167 ms) : 0, 20167 Remote Config [baseline] (641.269 µs) : 0, 641 Remote Config [candidate] (649.256 µs) : 0, 649 Telemetry [baseline] (7.998 ms) : 0, 7998 Telemetry [candidate] (9.044 ms) : 0, 9044 section iast BytebuddyAgent [baseline] (791.569 ms) : 0, 791569 BytebuddyAgent [candidate] (791.289 ms) : 0, 791289 GlobalTracer [baseline] (299.499 ms) : 0, 299499 GlobalTracer [candidate] (301.19 ms) : 0, 301190 AppSec [baseline] (52.232 ms) : 0, 52232 AppSec [candidate] (51.923 ms) : 0, 51923 IAST [baseline] (23.563 ms) : 0, 23563 IAST [candidate] (25.358 ms) : 0, 25358 Remote Config [baseline] (1.387 ms) : 0, 1387 Remote Config [candidate] (614.379 µs) : 0, 614 Telemetry [baseline] (7.343 ms) : 0, 7343 Telemetry [candidate] (7.326 ms) : 0, 7326 section profiling BytebuddyAgent [baseline] (673.132 ms) : 0, 673132 BytebuddyAgent [candidate] (678.268 ms) : 0, 678268 GlobalTracer [baseline] (394.044 ms) : 0, 394044 GlobalTracer [candidate] (396.984 ms) : 0, 396984 AppSec [baseline] (52.04 ms) : 0, 52040 AppSec [candidate] (52.534 ms) : 0, 52534 Remote Config [baseline] (697.467 µs) : 0, 697 Remote Config [candidate] (731.1 µs) : 0, 731 Telemetry [baseline] (7.404 ms) : 0, 7404 Telemetry [candidate] (7.45 ms) : 0, 7450 ProfilingAgent [baseline] (96.497 ms) : 0, 96497 ProfilingAgent [candidate] (95.736 ms) : 0, 95736 Profiling [baseline] (96.521 ms) : 0, 96521 Profiling [candidate] (95.761 ms) : 0, 95761 ```
Startup time reports for insecure-bank ```mermaid gantt title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368 dateFormat X axisFormat %s section tracing Agent [baseline] (1.063 s) : 0, 1063476 Total [baseline] (8.549 s) : 0, 8549165 Agent [candidate] (1.066 s) : 0, 1065991 Total [candidate] (8.557 s) : 0, 8556767 section iast Agent [baseline] (1.187 s) : 0, 1187172 Total [baseline] (8.988 s) : 0, 8988144 Agent [candidate] (1.187 s) : 0, 1186866 Total [candidate] (8.973 s) : 0, 8972514 section iast_HARDCODED_SECRET_DISABLED Agent [baseline] (1.198 s) : 0, 1197530 Total [baseline] (8.998 s) : 0, 8998454 Agent [candidate] (1.206 s) : 0, 1205825 Total [candidate] (9.002 s) : 0, 9001669 section iast_TELEMETRY_OFF Agent [baseline] (1.186 s) : 0, 1185599 Total [baseline] (8.982 s) : 0, 8981816 Agent [candidate] (1.194 s) : 0, 1193914 Total [candidate] (8.996 s) : 0, 8996441 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.063 s |-| |Agent|iast|1.187 s |123.696 ms (11.6%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.198 s |134.055 ms (12.6%)| |Agent|iast_TELEMETRY_OFF|1.186 s |122.123 ms (11.5%)| |Total|tracing|8.549 s |-| |Total|iast|8.988 s |438.979 ms (5.1%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.998 s |449.289 ms (5.3%)| |Total|iast_TELEMETRY_OFF|8.982 s |432.651 ms (5.1%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.066 s |-| |Agent|iast|1.187 s |120.875 ms (11.3%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.206 s |139.834 ms (13.1%)| |Agent|iast_TELEMETRY_OFF|1.194 s |127.923 ms (12.0%)| |Total|tracing|8.557 s |-| |Total|iast|8.973 s |415.748 ms (4.9%)| |Total|iast_HARDCODED_SECRET_DISABLED|9.002 s |444.903 ms (5.2%)| |Total|iast_TELEMETRY_OFF|8.996 s |439.675 ms (5.1%)| ```mermaid gantt title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368 dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (678.88 ms) : 0, 678880 BytebuddyAgent [candidate] (679.544 ms) : 0, 679544 GlobalTracer [baseline] (311.207 ms) : 0, 311207 GlobalTracer [candidate] (312.409 ms) : 0, 312409 AppSec [baseline] (51.496 ms) : 0, 51496 AppSec [candidate] (52.135 ms) : 0, 52135 Remote Config [baseline] (714.087 µs) : 0, 714 Remote Config [candidate] (722.944 µs) : 0, 723 Telemetry [baseline] (7.566 ms) : 0, 7566 Telemetry [candidate] (7.562 ms) : 0, 7562 section iast BytebuddyAgent [baseline] (789.528 ms) : 0, 789528 BytebuddyAgent [candidate] (789.541 ms) : 0, 789541 GlobalTracer [baseline] (299.523 ms) : 0, 299523 GlobalTracer [candidate] (299.769 ms) : 0, 299769 AppSec [baseline] (50.456 ms) : 0, 50456 AppSec [candidate] (53.37 ms) : 0, 53370 IAST [baseline] (26.116 ms) : 0, 26116 IAST [candidate] (22.679 ms) : 0, 22679 Remote Config [baseline] (644.366 µs) : 0, 644 Remote Config [candidate] (587.667 µs) : 0, 588 Telemetry [baseline] (7.275 ms) : 0, 7275 Telemetry [candidate] (7.301 ms) : 0, 7301 section iast_HARDCODED_SECRET_DISABLED BytebuddyAgent [baseline] (796.914 ms) : 0, 796914 BytebuddyAgent [candidate] (803.669 ms) : 0, 803669 GlobalTracer [baseline] (302.231 ms) : 0, 302231 GlobalTracer [candidate] (302.937 ms) : 0, 302937 AppSec [baseline] (55.565 ms) : 0, 55565 AppSec [candidate] (54.568 ms) : 0, 54568 IAST [baseline] (21.061 ms) : 0, 21061 IAST [candidate] (22.811 ms) : 0, 22811 Remote Config [baseline] (597.673 µs) : 0, 598 Remote Config [candidate] (598.947 µs) : 0, 599 Telemetry [baseline] (7.375 ms) : 0, 7375 Telemetry [candidate] (7.37 ms) : 0, 7370 section iast_TELEMETRY_OFF BytebuddyAgent [baseline] (787.837 ms) : 0, 787837 BytebuddyAgent [candidate] (793.691 ms) : 0, 793691 GlobalTracer [baseline] (300.197 ms) : 0, 300197 GlobalTracer [candidate] (301.855 ms) : 0, 301855 AppSec [baseline] (54.382 ms) : 0, 54382 AppSec [candidate] (55.789 ms) : 0, 55789 IAST [baseline] (21.616 ms) : 0, 21616 IAST [candidate] (20.88 ms) : 0, 20880 Remote Config [baseline] (611.028 µs) : 0, 611 Remote Config [candidate] (608.906 µs) : 0, 609 Telemetry [baseline] (7.26 ms) : 0, 7260 Telemetry [candidate] (7.334 ms) : 0, 7334 ```

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-11T14:29:56 2024-09-11T14:36:46
git_branch master smola/tomcat-async-flaky
git_commit_date 1726062980 1726064446
git_commit_sha 4abf841368 4f0abe12a2
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~4f0abe12a2
start_time 2024-09-11T14:29:43 2024-09-11T14:36:33
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1726065752 |1726065752 | |ci_job_id |635568524 |635568524 | |ci_pipeline_id|44081872 |44081872 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for insecure-bank ```mermaid gantt title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368 dateFormat X axisFormat %s section baseline no_agent (375.076 µs) : 355, 395 . : milestone, 375, iast (482.44 µs) : 461, 504 . : milestone, 482, iast_FULL (562.097 µs) : 541, 584 . : milestone, 562, iast_GLOBAL (508.695 µs) : 487, 531 . : milestone, 509, iast_HARDCODED_SECRET_DISABLED (483.104 µs) : 461, 505 . : milestone, 483, iast_INACTIVE (447.074 µs) : 426, 468 . : milestone, 447, iast_TELEMETRY_OFF (481.405 µs) : 459, 504 . : milestone, 481, tracing (441.139 µs) : 421, 461 . : milestone, 441, section candidate no_agent (379.076 µs) : 359, 399 . : milestone, 379, iast (481.721 µs) : 460, 504 . : milestone, 482, iast_FULL (556.225 µs) : 535, 577 . : milestone, 556, iast_GLOBAL (508.266 µs) : 487, 529 . : milestone, 508, iast_HARDCODED_SECRET_DISABLED (494.021 µs) : 473, 515 . : milestone, 494, iast_INACTIVE (446.813 µs) : 426, 467 . : milestone, 447, iast_TELEMETRY_OFF (476.836 µs) : 454, 500 . : milestone, 477, tracing (444.096 µs) : 423, 465 . : milestone, 444, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|375.076 µs [354.726 µs, 395.426 µs]|-| |iast|482.44 µs [461.247 µs, 503.632 µs]|107.364 µs (28.6%)| |iast_FULL|562.097 µs [540.692 µs, 583.503 µs]|187.021 µs (49.9%)| |iast_GLOBAL|508.695 µs [486.866 µs, 530.524 µs]|133.619 µs (35.6%)| |iast_HARDCODED_SECRET_DISABLED|483.104 µs [461.147 µs, 505.062 µs]|108.029 µs (28.8%)| |iast_INACTIVE|447.074 µs [426.285 µs, 467.863 µs]|71.998 µs (19.2%)| |iast_TELEMETRY_OFF|481.405 µs [458.629 µs, 504.181 µs]|106.329 µs (28.3%)| |tracing|441.139 µs [420.877 µs, 461.401 µs]|66.063 µs (17.6%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|379.076 µs [358.842 µs, 399.309 µs]|-| |iast|481.721 µs [459.932 µs, 503.51 µs]|102.645 µs (27.1%)| |iast_FULL|556.225 µs [535.051 µs, 577.399 µs]|177.149 µs (46.7%)| |iast_GLOBAL|508.266 µs [487.279 µs, 529.253 µs]|129.19 µs (34.1%)| |iast_HARDCODED_SECRET_DISABLED|494.021 µs [472.95 µs, 515.092 µs]|114.945 µs (30.3%)| |iast_INACTIVE|446.813 µs [426.23 µs, 467.396 µs]|67.737 µs (17.9%)| |iast_TELEMETRY_OFF|476.836 µs [454.101 µs, 499.572 µs]|97.76 µs (25.8%)| |tracing|444.096 µs [423.331 µs, 464.861 µs]|65.02 µs (17.2%)|
Request duration reports for petclinic ```mermaid gantt title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368 dateFormat X axisFormat %s section baseline no_agent (1.339 ms) : 1320, 1358 . : milestone, 1339, appsec (1.721 ms) : 1696, 1746 . : milestone, 1721, appsec_no_iast (1.704 ms) : 1679, 1729 . : milestone, 1704, iast (1.473 ms) : 1450, 1496 . : milestone, 1473, profiling (1.47 ms) : 1446, 1495 . : milestone, 1470, tracing (1.44 ms) : 1414, 1465 . : milestone, 1440, section candidate no_agent (1.34 ms) : 1322, 1359 . : milestone, 1340, appsec (1.716 ms) : 1691, 1741 . : milestone, 1716, appsec_no_iast (1.737 ms) : 1713, 1761 . : milestone, 1737, iast (1.47 ms) : 1447, 1493 . : milestone, 1470, profiling (1.491 ms) : 1467, 1515 . : milestone, 1491, tracing (1.45 ms) : 1425, 1475 . : milestone, 1450, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.339 ms [1.32 ms, 1.358 ms]|-| |appsec|1.721 ms [1.696 ms, 1.746 ms]|382.181 µs (28.5%)| |appsec_no_iast|1.704 ms [1.679 ms, 1.729 ms]|365.087 µs (27.3%)| |iast|1.473 ms [1.45 ms, 1.496 ms]|134.457 µs (10.0%)| |profiling|1.47 ms [1.446 ms, 1.495 ms]|131.365 µs (9.8%)| |tracing|1.44 ms [1.414 ms, 1.465 ms]|100.816 µs (7.5%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.34 ms [1.322 ms, 1.359 ms]|-| |appsec|1.716 ms [1.691 ms, 1.741 ms]|375.823 µs (28.0%)| |appsec_no_iast|1.737 ms [1.713 ms, 1.761 ms]|396.455 µs (29.6%)| |iast|1.47 ms [1.447 ms, 1.493 ms]|129.557 µs (9.7%)| |profiling|1.491 ms [1.467 ms, 1.515 ms]|150.661 µs (11.2%)| |tracing|1.45 ms [1.425 ms, 1.475 ms]|109.616 µs (8.2%)|

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/tomcat-async-flaky
git_commit_date 1726062980 1726064446
git_commit_sha 4abf841368 4f0abe12a2
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~4f0abe12a2
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |biojava |biojava | |ci_job_date |1726066397 |1726066397 | |ci_job_id |635568525 |635568525 | |ci_pipeline_id|44081872 |44081872 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |appsec |appsec |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat ```mermaid gantt title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368 dateFormat X axisFormat %s section baseline no_agent (1.463 ms) : 1452, 1475 . : milestone, 1463, appsec (2.281 ms) : 2240, 2321 . : milestone, 2281, iast (2.024 ms) : 1976, 2072 . : milestone, 2024, iast_GLOBAL (2.099 ms) : 2048, 2150 . : milestone, 2099, profiling (1.931 ms) : 1889, 1972 . : milestone, 1931, tracing (1.901 ms) : 1862, 1939 . : milestone, 1901, section candidate no_agent (1.458 ms) : 1447, 1470 . : milestone, 1458, appsec (2.288 ms) : 2248, 2329 . : milestone, 2288, iast (2.034 ms) : 1986, 2083 . : milestone, 2034, iast_GLOBAL (2.074 ms) : 2025, 2124 . : milestone, 2074, profiling (1.912 ms) : 1872, 1952 . : milestone, 1912, tracing (1.882 ms) : 1844, 1919 . : milestone, 1882, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.463 ms [1.452 ms, 1.475 ms]|-| |appsec|2.281 ms [2.24 ms, 2.321 ms]|817.563 µs (55.9%)| |iast|2.024 ms [1.976 ms, 2.072 ms]|560.575 µs (38.3%)| |iast_GLOBAL|2.099 ms [2.048 ms, 2.15 ms]|635.919 µs (43.5%)| |profiling|1.931 ms [1.889 ms, 1.972 ms]|467.541 µs (32.0%)| |tracing|1.901 ms [1.862 ms, 1.939 ms]|437.561 µs (29.9%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.458 ms [1.447 ms, 1.47 ms]|-| |appsec|2.288 ms [2.248 ms, 2.329 ms]|829.78 µs (56.9%)| |iast|2.034 ms [1.986 ms, 2.083 ms]|576.086 µs (39.5%)| |iast_GLOBAL|2.074 ms [2.025 ms, 2.124 ms]|615.811 µs (42.2%)| |profiling|1.912 ms [1.872 ms, 1.952 ms]|453.75 µs (31.1%)| |tracing|1.882 ms [1.844 ms, 1.919 ms]|423.163 µs (29.0%)|
Execution time for biojava ```mermaid gantt title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368 dateFormat X axisFormat %s section baseline no_agent (14.977 s) : 14977000, 14977000 . : milestone, 14977000, appsec (15.35 s) : 15350000, 15350000 . : milestone, 15350000, iast (18.919 s) : 18919000, 18919000 . : milestone, 18919000, iast_GLOBAL (18.232 s) : 18232000, 18232000 . : milestone, 18232000, profiling (16.034 s) : 16034000, 16034000 . : milestone, 16034000, tracing (15.103 s) : 15103000, 15103000 . : milestone, 15103000, section candidate no_agent (15.429 s) : 15429000, 15429000 . : milestone, 15429000, appsec (15.069 s) : 15069000, 15069000 . : milestone, 15069000, iast (18.856 s) : 18856000, 18856000 . : milestone, 18856000, iast_GLOBAL (18.001 s) : 18001000, 18001000 . : milestone, 18001000, profiling (15.446 s) : 15446000, 15446000 . : milestone, 15446000, tracing (14.953 s) : 14953000, 14953000 . : milestone, 14953000, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|14.977 s [14.977 s, 14.977 s]|-| |appsec|15.35 s [15.35 s, 15.35 s]|373.0 ms (2.5%)| |iast|18.919 s [18.919 s, 18.919 s]|3.942 s (26.3%)| |iast_GLOBAL|18.232 s [18.232 s, 18.232 s]|3.255 s (21.7%)| |profiling|16.034 s [16.034 s, 16.034 s]|1.057 s (7.1%)| |tracing|15.103 s [15.103 s, 15.103 s]|126.0 ms (0.8%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.429 s [15.429 s, 15.429 s]|-| |appsec|15.069 s [15.069 s, 15.069 s]|-360.0 ms (-2.3%)| |iast|18.856 s [18.856 s, 18.856 s]|3.427 s (22.2%)| |iast_GLOBAL|18.001 s [18.001 s, 18.001 s]|2.572 s (16.7%)| |profiling|15.446 s [15.446 s, 15.446 s]|17.0 ms (0.1%)| |tracing|14.953 s [14.953 s, 14.953 s]|-476.0 ms (-3.1%)|
PerfectSlayer commented 3 weeks ago

Is this PR still not to merge? (according the do not merge tag)

smola commented 2 weeks ago

@PerfectSlayer I was waiting to confirm this actually fixed the issue. I now think it does, also fixed an additional test with the same issue. Good to merge now if it passes CI.