search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
572 stars 284 forks source link

fix: add system tests aws credentials #7629

Open wconti27 opened 1 week ago

wconti27 commented 1 week ago

What Does This Do

add aws credentials for new system tests that create aws resources

Motivation

more testing, more fun

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

pr-commenter[bot] commented 1 week ago

Benchmarks

Startup

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-26T13:13:17 2024-09-26T13:20:07
git_branch master conti/system-tests-aws-access
git_commit_date 1727340475 1727355548
git_commit_sha 8ee4a5d6dd c5ea23c844
release_version 1.40.0-SNAPSHOT~8ee4a5d6dd 1.40.0-SNAPSHOT~c5ea23c844
start_time 2024-09-26T13:13:03 2024-09-26T13:19:53
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1727357152 |1727357152 | |ci_job_id |652591133 |652591133 | |ci_pipeline_id|45207160 |45207160 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic ```mermaid gantt title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~c5ea23c844, baseline=1.40.0-SNAPSHOT~8ee4a5d6dd dateFormat X axisFormat %s section baseline no_agent (1.336 ms) : 1316, 1356 . : milestone, 1336, appsec (1.733 ms) : 1709, 1757 . : milestone, 1733, appsec_no_iast (1.706 ms) : 1681, 1730 . : milestone, 1706, iast (1.477 ms) : 1455, 1500 . : milestone, 1477, profiling (1.462 ms) : 1438, 1487 . : milestone, 1462, tracing (1.452 ms) : 1428, 1477 . : milestone, 1452, section candidate no_agent (1.331 ms) : 1312, 1350 . : milestone, 1331, appsec (1.717 ms) : 1692, 1741 . : milestone, 1717, appsec_no_iast (1.716 ms) : 1693, 1740 . : milestone, 1716, iast (1.499 ms) : 1476, 1521 . : milestone, 1499, profiling (1.481 ms) : 1456, 1505 . : milestone, 1481, tracing (1.473 ms) : 1448, 1497 . : milestone, 1473, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.336 ms [1.316 ms, 1.356 ms]|-| |appsec|1.733 ms [1.709 ms, 1.757 ms]|396.893 µs (29.7%)| |appsec_no_iast|1.706 ms [1.681 ms, 1.73 ms]|369.795 µs (27.7%)| |iast|1.477 ms [1.455 ms, 1.5 ms]|141.494 µs (10.6%)| |profiling|1.462 ms [1.438 ms, 1.487 ms]|126.267 µs (9.5%)| |tracing|1.452 ms [1.428 ms, 1.477 ms]|116.405 µs (8.7%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.331 ms [1.312 ms, 1.35 ms]|-| |appsec|1.717 ms [1.692 ms, 1.741 ms]|385.596 µs (29.0%)| |appsec_no_iast|1.716 ms [1.693 ms, 1.74 ms]|385.305 µs (28.9%)| |iast|1.499 ms [1.476 ms, 1.521 ms]|167.519 µs (12.6%)| |profiling|1.481 ms [1.456 ms, 1.505 ms]|149.729 µs (11.2%)| |tracing|1.473 ms [1.448 ms, 1.497 ms]|141.553 µs (10.6%)|
Request duration reports for insecure-bank ```mermaid gantt title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~c5ea23c844, baseline=1.40.0-SNAPSHOT~8ee4a5d6dd dateFormat X axisFormat %s section baseline no_agent (373.488 µs) : 352, 395 . : milestone, 373, iast (482.04 µs) : 461, 503 . : milestone, 482, iast_FULL (560.496 µs) : 539, 582 . : milestone, 560, iast_GLOBAL (512.925 µs) : 491, 535 . : milestone, 513, iast_HARDCODED_SECRET_DISABLED (486.456 µs) : 465, 508 . : milestone, 486, iast_INACTIVE (455.039 µs) : 433, 477 . : milestone, 455, iast_TELEMETRY_OFF (490.286 µs) : 466, 514 . : milestone, 490, tracing (445.293 µs) : 425, 466 . : milestone, 445, section candidate no_agent (373.27 µs) : 353, 393 . : milestone, 373, iast (483.708 µs) : 462, 505 . : milestone, 484, iast_FULL (553.101 µs) : 532, 575 . : milestone, 553, iast_GLOBAL (517.668 µs) : 495, 540 . : milestone, 518, iast_HARDCODED_SECRET_DISABLED (489.123 µs) : 468, 511 . : milestone, 489, iast_INACTIVE (455.522 µs) : 435, 476 . : milestone, 456, iast_TELEMETRY_OFF (480.394 µs) : 457, 504 . : milestone, 480, tracing (444.742 µs) : 425, 465 . : milestone, 445, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|373.488 µs [352.476 µs, 394.501 µs]|-| |iast|482.04 µs [461.212 µs, 502.868 µs]|108.551 µs (29.1%)| |iast_FULL|560.496 µs [538.831 µs, 582.16 µs]|187.007 µs (50.1%)| |iast_GLOBAL|512.925 µs [490.995 µs, 534.855 µs]|139.437 µs (37.3%)| |iast_HARDCODED_SECRET_DISABLED|486.456 µs [465.266 µs, 507.646 µs]|112.967 µs (30.2%)| |iast_INACTIVE|455.039 µs [433.395 µs, 476.684 µs]|81.551 µs (21.8%)| |iast_TELEMETRY_OFF|490.286 µs [466.429 µs, 514.144 µs]|116.798 µs (31.3%)| |tracing|445.293 µs [424.528 µs, 466.057 µs]|71.804 µs (19.2%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|373.27 µs [353.409 µs, 393.13 µs]|-| |iast|483.708 µs [462.312 µs, 505.105 µs]|110.439 µs (29.6%)| |iast_FULL|553.101 µs [531.693 µs, 574.508 µs]|179.831 µs (48.2%)| |iast_GLOBAL|517.668 µs [494.968 µs, 540.368 µs]|144.399 µs (38.7%)| |iast_HARDCODED_SECRET_DISABLED|489.123 µs [467.642 µs, 510.603 µs]|115.853 µs (31.0%)| |iast_INACTIVE|455.522 µs [434.958 µs, 476.086 µs]|82.252 µs (22.0%)| |iast_TELEMETRY_OFF|480.394 µs [457.16 µs, 503.629 µs]|107.125 µs (28.7%)| |tracing|444.742 µs [424.856 µs, 464.628 µs]|71.472 µs (19.1%)|

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master conti/system-tests-aws-access
git_commit_date 1727340475 1727355548
git_commit_sha 8ee4a5d6dd c5ea23c844
release_version 1.40.0-SNAPSHOT~8ee4a5d6dd 1.40.0-SNAPSHOT~c5ea23c844
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |biojava |biojava | |ci_job_date |1727357713 |1727357713 | |ci_job_id |652591134 |652591134 | |ci_pipeline_id|45207160 |45207160 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |appsec |appsec |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat ```mermaid gantt title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~c5ea23c844, baseline=1.40.0-SNAPSHOT~8ee4a5d6dd dateFormat X axisFormat %s section baseline no_agent (1.461 ms) : 1450, 1472 . : milestone, 1461, appsec (2.3 ms) : 2259, 2340 . : milestone, 2300, iast (2.063 ms) : 2012, 2114 . : milestone, 2063, iast_GLOBAL (2.091 ms) : 2041, 2142 . : milestone, 2091, profiling (1.922 ms) : 1881, 1962 . : milestone, 1922, tracing (1.899 ms) : 1860, 1938 . : milestone, 1899, section candidate no_agent (1.455 ms) : 1444, 1467 . : milestone, 1455, appsec (2.305 ms) : 2265, 2345 . : milestone, 2305, iast (2.05 ms) : 2000, 2101 . : milestone, 2050, iast_GLOBAL (2.091 ms) : 2041, 2142 . : milestone, 2091, profiling (2.41 ms) : 2224, 2597 . : milestone, 2410, tracing (1.902 ms) : 1864, 1941 . : milestone, 1902, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.461 ms [1.45 ms, 1.472 ms]|-| |appsec|2.3 ms [2.259 ms, 2.34 ms]|838.94 µs (57.4%)| |iast|2.063 ms [2.012 ms, 2.114 ms]|601.716 µs (41.2%)| |iast_GLOBAL|2.091 ms [2.041 ms, 2.142 ms]|630.357 µs (43.1%)| |profiling|1.922 ms [1.881 ms, 1.962 ms]|460.695 µs (31.5%)| |tracing|1.899 ms [1.86 ms, 1.938 ms]|438.006 µs (30.0%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.455 ms [1.444 ms, 1.467 ms]|-| |appsec|2.305 ms [2.265 ms, 2.345 ms]|849.362 µs (58.4%)| |iast|2.05 ms [2.0 ms, 2.101 ms]|594.999 µs (40.9%)| |iast_GLOBAL|2.091 ms [2.041 ms, 2.142 ms]|635.791 µs (43.7%)| |profiling|2.41 ms [2.224 ms, 2.597 ms]|954.9 µs (65.6%)| |tracing|1.902 ms [1.864 ms, 1.941 ms]|446.589 µs (30.7%)|
Execution time for biojava ```mermaid gantt title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~c5ea23c844, baseline=1.40.0-SNAPSHOT~8ee4a5d6dd dateFormat X axisFormat %s section baseline no_agent (15.689 s) : 15689000, 15689000 . : milestone, 15689000, appsec (15.121 s) : 15121000, 15121000 . : milestone, 15121000, iast (18.625 s) : 18625000, 18625000 . : milestone, 18625000, iast_GLOBAL (18.308 s) : 18308000, 18308000 . : milestone, 18308000, profiling (15.154 s) : 15154000, 15154000 . : milestone, 15154000, tracing (15.413 s) : 15413000, 15413000 . : milestone, 15413000, section candidate no_agent (14.901 s) : 14901000, 14901000 . : milestone, 14901000, appsec (15.14 s) : 15140000, 15140000 . : milestone, 15140000, iast (18.821 s) : 18821000, 18821000 . : milestone, 18821000, iast_GLOBAL (18.075 s) : 18075000, 18075000 . : milestone, 18075000, profiling (15.402 s) : 15402000, 15402000 . : milestone, 15402000, tracing (15.27 s) : 15270000, 15270000 . : milestone, 15270000, ``` * **baseline** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|15.689 s [15.689 s, 15.689 s]|-| |appsec|15.121 s [15.121 s, 15.121 s]|-568.0 ms (-3.6%)| |iast|18.625 s [18.625 s, 18.625 s]|2.936 s (18.7%)| |iast_GLOBAL|18.308 s [18.308 s, 18.308 s]|2.619 s (16.7%)| |profiling|15.154 s [15.154 s, 15.154 s]|-535.0 ms (-3.4%)| |tracing|15.413 s [15.413 s, 15.413 s]|-276.0 ms (-1.8%)| * **candidate** results |Variant|Execution Time [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|14.901 s [14.901 s, 14.901 s]|-| |appsec|15.14 s [15.14 s, 15.14 s]|239.0 ms (1.6%)| |iast|18.821 s [18.821 s, 18.821 s]|3.92 s (26.3%)| |iast_GLOBAL|18.075 s [18.075 s, 18.075 s]|3.174 s (21.3%)| |profiling|15.402 s [15.402 s, 15.402 s]|501.0 ms (3.4%)| |tracing|15.27 s [15.27 s, 15.27 s]|369.0 ms (2.5%)|