search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
573 stars 284 forks source link

Improve milestone automation #7553

Closed PerfectSlayer closed 2 weeks ago

PerfectSlayer commented 3 weeks ago

What Does This Do

This PR improves milestone automation by:

Motivation

Remove deprecated third party actions Add support for patch versions Add missing permissions

Additional Notes

Related to https://github.com/DataDog/dd-trace-java/pull/7531

Contributor Checklist

Jira ticket: VULN-8166

pr-commenter[bot] commented 3 weeks ago

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/gh-workflow
git_commit_date 1725979768 1725984516
git_commit_sha 04db2f0bfb 724ce01c8c
release_version 1.40.0-SNAPSHOT~04db2f0bfb 1.40.0-SNAPSHOT~724ce01c8c
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1725987026 |1725987026 | |ci_job_id |634142100 |634142100 | |ci_pipeline_id|43998899 |43998899 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |module |Agent |Agent | |parent |None |None | |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for petclinic ```mermaid gantt title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb dateFormat X axisFormat %s section tracing Agent [baseline] (1.064 s) : 0, 1064080 Total [baseline] (10.385 s) : 0, 10384989 Agent [candidate] (1.063 s) : 0, 1062601 Total [candidate] (10.347 s) : 0, 10346776 section appsec Agent [baseline] (1.199 s) : 0, 1198994 Total [baseline] (10.614 s) : 0, 10613793 Agent [candidate] (1.198 s) : 0, 1198053 Total [candidate] (10.631 s) : 0, 10630767 section iast Agent [baseline] (1.19 s) : 0, 1190261 Total [baseline] (10.828 s) : 0, 10827658 Agent [candidate] (1.189 s) : 0, 1188690 Total [candidate] (10.888 s) : 0, 10887784 section profiling Agent [baseline] (1.268 s) : 0, 1268255 Total [baseline] (10.712 s) : 0, 10712410 Agent [candidate] (1.258 s) : 0, 1258489 Total [candidate] (10.563 s) : 0, 10563425 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.064 s |-| |Agent|appsec|1.199 s |134.914 ms (12.7%)| |Agent|iast|1.19 s |126.181 ms (11.9%)| |Agent|profiling|1.268 s |204.175 ms (19.2%)| |Total|tracing|10.385 s |-| |Total|appsec|10.614 s |228.805 ms (2.2%)| |Total|iast|10.828 s |442.669 ms (4.3%)| |Total|profiling|10.712 s |327.421 ms (3.2%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.063 s |-| |Agent|appsec|1.198 s |135.452 ms (12.7%)| |Agent|iast|1.189 s |126.089 ms (11.9%)| |Agent|profiling|1.258 s |195.889 ms (18.4%)| |Total|tracing|10.347 s |-| |Total|appsec|10.631 s |283.991 ms (2.7%)| |Total|iast|10.888 s |541.008 ms (5.2%)| |Total|profiling|10.563 s |216.649 ms (2.1%)| ```mermaid gantt title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (679.042 ms) : 0, 679042 BytebuddyAgent [candidate] (678.137 ms) : 0, 678137 GlobalTracer [baseline] (311.61 ms) : 0, 311610 GlobalTracer [candidate] (310.525 ms) : 0, 310525 AppSec [baseline] (51.564 ms) : 0, 51564 AppSec [candidate] (51.966 ms) : 0, 51966 Remote Config [baseline] (694.164 µs) : 0, 694 Remote Config [candidate] (695.594 µs) : 0, 696 Telemetry [baseline] (7.547 ms) : 0, 7547 Telemetry [candidate] (7.578 ms) : 0, 7578 section appsec BytebuddyAgent [baseline] (699.905 ms) : 0, 699905 BytebuddyAgent [candidate] (697.598 ms) : 0, 697598 GlobalTracer [baseline] (304.544 ms) : 0, 304544 GlobalTracer [candidate] (304.592 ms) : 0, 304592 AppSec [baseline] (160.748 ms) : 0, 160748 AppSec [candidate] (161.521 ms) : 0, 161521 Remote Config [baseline] (646.913 µs) : 0, 647 Remote Config [candidate] (633.739 µs) : 0, 634 Telemetry [baseline] (8.311 ms) : 0, 8311 Telemetry [candidate] (8.98 ms) : 0, 8980 IAST [baseline] (22.377 ms) : 0, 22377 IAST [candidate] (22.182 ms) : 0, 22182 section iast BytebuddyAgent [baseline] (791.572 ms) : 0, 791572 BytebuddyAgent [candidate] (790.335 ms) : 0, 790335 GlobalTracer [baseline] (299.94 ms) : 0, 299940 GlobalTracer [candidate] (299.541 ms) : 0, 299541 AppSec [baseline] (55.009 ms) : 0, 55009 AppSec [candidate] (52.573 ms) : 0, 52573 Remote Config [baseline] (600.536 µs) : 0, 601 Remote Config [candidate] (607.991 µs) : 0, 608 Telemetry [baseline] (7.433 ms) : 0, 7433 Telemetry [candidate] (9.762 ms) : 0, 9762 IAST [baseline] (22.035 ms) : 0, 22035 IAST [candidate] (22.205 ms) : 0, 22205 section profiling BytebuddyAgent [baseline] (676.943 ms) : 0, 676943 BytebuddyAgent [candidate] (671.115 ms) : 0, 671115 GlobalTracer [baseline] (395.46 ms) : 0, 395460 GlobalTracer [candidate] (393.02 ms) : 0, 393020 AppSec [baseline] (52.605 ms) : 0, 52605 AppSec [candidate] (52.196 ms) : 0, 52196 Remote Config [baseline] (712.621 µs) : 0, 713 Remote Config [candidate] (703.625 µs) : 0, 704 Telemetry [baseline] (7.43 ms) : 0, 7430 Telemetry [candidate] (7.395 ms) : 0, 7395 ProfilingAgent [baseline] (96.919 ms) : 0, 96919 ProfilingAgent [candidate] (96.076 ms) : 0, 96076 Profiling [baseline] (96.943 ms) : 0, 96943 Profiling [candidate] (96.1 ms) : 0, 96100 ```
Startup time reports for insecure-bank ```mermaid gantt title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb dateFormat X axisFormat %s section tracing Agent [baseline] (1.064 s) : 0, 1064220 Total [baseline] (8.532 s) : 0, 8532344 Agent [candidate] (1.069 s) : 0, 1068799 Total [candidate] (8.538 s) : 0, 8537542 section iast Agent [baseline] (1.188 s) : 0, 1188191 Total [baseline] (8.976 s) : 0, 8975813 Agent [candidate] (1.195 s) : 0, 1194699 Total [candidate] (9.042 s) : 0, 9042473 section iast_HARDCODED_SECRET_DISABLED Agent [baseline] (1.188 s) : 0, 1187641 Total [baseline] (8.967 s) : 0, 8967156 Agent [candidate] (1.189 s) : 0, 1188851 Total [candidate] (8.962 s) : 0, 8962274 section iast_TELEMETRY_OFF Agent [baseline] (1.193 s) : 0, 1193448 Total [baseline] (8.984 s) : 0, 8983736 Agent [candidate] (1.183 s) : 0, 1183366 Total [candidate] (9.024 s) : 0, 9024140 ``` * **baseline** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.064 s |-| |Agent|iast|1.188 s |123.971 ms (11.6%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.188 s |123.421 ms (11.6%)| |Agent|iast_TELEMETRY_OFF|1.193 s |129.228 ms (12.1%)| |Total|tracing|8.532 s |-| |Total|iast|8.976 s |443.469 ms (5.2%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.967 s |434.813 ms (5.1%)| |Total|iast_TELEMETRY_OFF|8.984 s |451.393 ms (5.3%)| * **candidate** results |Module|Variant|Duration|Δ tracing| |---|---|---|---| |Agent|tracing|1.069 s |-| |Agent|iast|1.195 s |125.9 ms (11.8%)| |Agent|iast_HARDCODED_SECRET_DISABLED|1.189 s |120.052 ms (11.2%)| |Agent|iast_TELEMETRY_OFF|1.183 s |114.567 ms (10.7%)| |Total|tracing|8.538 s |-| |Total|iast|9.042 s |504.932 ms (5.9%)| |Total|iast_HARDCODED_SECRET_DISABLED|8.962 s |424.733 ms (5.0%)| |Total|iast_TELEMETRY_OFF|9.024 s |486.598 ms (5.7%)| ```mermaid gantt title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb dateFormat X axisFormat %s section tracing BytebuddyAgent [baseline] (679.067 ms) : 0, 679067 BytebuddyAgent [candidate] (682.703 ms) : 0, 682703 GlobalTracer [baseline] (311.419 ms) : 0, 311419 GlobalTracer [candidate] (312.087 ms) : 0, 312087 AppSec [baseline] (51.857 ms) : 0, 51857 AppSec [candidate] (52.006 ms) : 0, 52006 Remote Config [baseline] (714.874 µs) : 0, 715 Remote Config [candidate] (692.386 µs) : 0, 692 Telemetry [baseline] (7.554 ms) : 0, 7554 Telemetry [candidate] (7.586 ms) : 0, 7586 section iast BytebuddyAgent [baseline] (790.873 ms) : 0, 790873 BytebuddyAgent [candidate] (794.772 ms) : 0, 794772 GlobalTracer [baseline] (299.241 ms) : 0, 299241 GlobalTracer [candidate] (301.169 ms) : 0, 301169 AppSec [baseline] (52.092 ms) : 0, 52092 AppSec [candidate] (53.271 ms) : 0, 53271 IAST [baseline] (23.655 ms) : 0, 23655 IAST [candidate] (22.906 ms) : 0, 22906 Remote Config [baseline] (1.39 ms) : 0, 1390 Remote Config [candidate] (597.984 µs) : 0, 598 Telemetry [baseline] (7.312 ms) : 0, 7312 Telemetry [candidate] (8.206 ms) : 0, 8206 section iast_HARDCODED_SECRET_DISABLED BytebuddyAgent [baseline] (790.948 ms) : 0, 790948 BytebuddyAgent [candidate] (790.283 ms) : 0, 790283 GlobalTracer [baseline] (300.03 ms) : 0, 300030 GlobalTracer [candidate] (300.158 ms) : 0, 300158 AppSec [baseline] (53.136 ms) : 0, 53136 AppSec [candidate] (53.894 ms) : 0, 53894 IAST [baseline] (21.134 ms) : 0, 21134 IAST [candidate] (22.059 ms) : 0, 22059 Remote Config [baseline] (609.239 µs) : 0, 609 Remote Config [candidate] (610.437 µs) : 0, 610 Telemetry [baseline] (8.153 ms) : 0, 8153 Telemetry [candidate] (8.165 ms) : 0, 8165 section iast_TELEMETRY_OFF BytebuddyAgent [baseline] (793.296 ms) : 0, 793296 BytebuddyAgent [candidate] (786.387 ms) : 0, 786387 GlobalTracer [baseline] (301.751 ms) : 0, 301751 GlobalTracer [candidate] (299.511 ms) : 0, 299511 AppSec [baseline] (55.99 ms) : 0, 55990 AppSec [candidate] (53.732 ms) : 0, 53732 IAST [baseline] (20.736 ms) : 0, 20736 IAST [candidate] (21.54 ms) : 0, 21540 Remote Config [baseline] (592.558 µs) : 0, 593 Remote Config [candidate] (591.787 µs) : 0, 592 Telemetry [baseline] (7.346 ms) : 0, 7346 Telemetry [candidate] (7.918 ms) : 0, 7918 ```

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-10T16:21:02 2024-09-10T16:27:51
git_branch master bbujon/gh-workflow
git_commit_date 1725979768 1725984516
git_commit_sha 04db2f0bfb 724ce01c8c
release_version 1.40.0-SNAPSHOT~04db2f0bfb 1.40.0-SNAPSHOT~724ce01c8c
start_time 2024-09-10T16:20:48 2024-09-10T16:27:38
See matching parameters | |Baseline |Candidate | |---------------------|----------------------------------------------|----------------------------------------------| |application |insecure-bank |insecure-bank | |ci_job_date |1725986017 |1725986017 | |ci_job_id |634142103 |634142103 | |ci_pipeline_id|43998899 |43998899 | |cpu_model |Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz|Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz| |variant |iast |iast |

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 19 unstable metrics.

Request duration reports for petclinic ```mermaid gantt title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb dateFormat X axisFormat %s section baseline no_agent (1.346 ms) : 1327, 1366 . : milestone, 1346, appsec (1.707 ms) : 1683, 1730 . : milestone, 1707, appsec_no_iast (1.726 ms) : 1702, 1750 . : milestone, 1726, iast (1.477 ms) : 1454, 1499 . : milestone, 1477, profiling (1.476 ms) : 1452, 1499 . : milestone, 1476, tracing (1.471 ms) : 1447, 1495 . : milestone, 1471, section candidate no_agent (1.33 ms) : 1311, 1350 . : milestone, 1330, appsec (1.729 ms) : 1705, 1753 . : milestone, 1729, appsec_no_iast (1.713 ms) : 1689, 1738 . : milestone, 1713, iast (1.462 ms) : 1439, 1485 . : milestone, 1462, profiling (1.52 ms) : 1495, 1546 . : milestone, 1520, tracing (1.459 ms) : 1434, 1484 . : milestone, 1459, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.346 ms [1.327 ms, 1.366 ms]|-| |appsec|1.707 ms [1.683 ms, 1.73 ms]|360.224 µs (26.8%)| |appsec_no_iast|1.726 ms [1.702 ms, 1.75 ms]|379.675 µs (28.2%)| |iast|1.477 ms [1.454 ms, 1.499 ms]|130.339 µs (9.7%)| |profiling|1.476 ms [1.452 ms, 1.499 ms]|129.339 µs (9.6%)| |tracing|1.471 ms [1.447 ms, 1.495 ms]|124.623 µs (9.3%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|1.33 ms [1.311 ms, 1.35 ms]|-| |appsec|1.729 ms [1.705 ms, 1.753 ms]|398.541 µs (30.0%)| |appsec_no_iast|1.713 ms [1.689 ms, 1.738 ms]|382.907 µs (28.8%)| |iast|1.462 ms [1.439 ms, 1.485 ms]|131.744 µs (9.9%)| |profiling|1.52 ms [1.495 ms, 1.546 ms]|190.015 µs (14.3%)| |tracing|1.459 ms [1.434 ms, 1.484 ms]|128.423 µs (9.7%)|
Request duration reports for insecure-bank ```mermaid gantt title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~724ce01c8c, baseline=1.40.0-SNAPSHOT~04db2f0bfb dateFormat X axisFormat %s section baseline no_agent (360.801 µs) : 341, 380 . : milestone, 361, iast (477.787 µs) : 457, 499 . : milestone, 478, iast_FULL (555.376 µs) : 534, 576 . : milestone, 555, iast_GLOBAL (512.497 µs) : 491, 534 . : milestone, 512, iast_HARDCODED_SECRET_DISABLED (480.115 µs) : 459, 502 . : milestone, 480, iast_INACTIVE (445.988 µs) : 425, 467 . : milestone, 446, iast_TELEMETRY_OFF (472.062 µs) : 450, 495 . : milestone, 472, tracing (444.89 µs) : 424, 466 . : milestone, 445, section candidate no_agent (363.672 µs) : 344, 383 . : milestone, 364, iast (478.806 µs) : 457, 501 . : milestone, 479, iast_FULL (553.956 µs) : 533, 575 . : milestone, 554, iast_GLOBAL (515.731 µs) : 493, 538 . : milestone, 516, iast_HARDCODED_SECRET_DISABLED (491.19 µs) : 468, 514 . : milestone, 491, iast_INACTIVE (440.292 µs) : 420, 461 . : milestone, 440, iast_TELEMETRY_OFF (472.177 µs) : 449, 495 . : milestone, 472, tracing (441.321 µs) : 421, 462 . : milestone, 441, ``` * **baseline** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|360.801 µs [341.319 µs, 380.282 µs]|-| |iast|477.787 µs [456.767 µs, 498.806 µs]|116.986 µs (32.4%)| |iast_FULL|555.376 µs [534.312 µs, 576.44 µs]|194.576 µs (53.9%)| |iast_GLOBAL|512.497 µs [490.553 µs, 534.441 µs]|151.696 µs (42.0%)| |iast_HARDCODED_SECRET_DISABLED|480.115 µs [458.709 µs, 501.521 µs]|119.314 µs (33.1%)| |iast_INACTIVE|445.988 µs [425.054 µs, 466.922 µs]|85.187 µs (23.6%)| |iast_TELEMETRY_OFF|472.062 µs [449.622 µs, 494.502 µs]|111.261 µs (30.8%)| |tracing|444.89 µs [423.535 µs, 466.245 µs]|84.089 µs (23.3%)| * **candidate** results |Variant|Request duration [CI 0.99]|Δ no_agent| |---|---|---| |no_agent|363.672 µs [343.956 µs, 383.388 µs]|-| |iast|478.806 µs [457.064 µs, 500.549 µs]|115.135 µs (31.7%)| |iast_FULL|553.956 µs [532.64 µs, 575.273 µs]|190.285 µs (52.3%)| |iast_GLOBAL|515.731 µs [493.361 µs, 538.101 µs]|152.06 µs (41.8%)| |iast_HARDCODED_SECRET_DISABLED|491.19 µs [467.959 µs, 514.421 µs]|127.519 µs (35.1%)| |iast_INACTIVE|440.292 µs [419.608 µs, 460.976 µs]|76.621 µs (21.1%)| |iast_TELEMETRY_OFF|472.177 µs [449.49 µs, 494.864 µs]|108.505 µs (29.8%)| |tracing|441.321 µs [420.939 µs, 461.703 µs]|77.649 µs (21.4%)|

Dacapo