moby / buildkit-bench

Set of benchmarks for BuildKit
https://moby.github.io/buildkit-bench/
Apache License 2.0
0 stars 2 forks source link

list candidates using GitHub API #10

Closed crazy-max closed 2 months ago

crazy-max commented 2 months ago

Instead of using hardcoded references we should have a dynamic list of ref candidates. We want a list of:

Example for 7 last days commits, 3 last releases and master as custom ref:

 {
  "refs": {
    "master": "11a4a07060b6284d79e4e9225c8c6c8b0a739cb9"
  },
  "releases": {
    "v0.13.0": "2afc050d57d17983f3f662d5424c2725a35c60f4",
    "v0.13.2": "2e18d709fefdcc2db20853ee241c75b058189d39",
    "v0.14.0": "4d9a4e5df9e11596a3261c1952cde3c6346be762",
    "v0.14.1": "eb864a84592468ee9b434326cb7efd66f58555af",
    "v0.15.0": "e83d79a51fb49aeb921d8a2348ae14a58701c98c",
    "v0.15.2": "9e14164a1099d3e41b58fc879cbdd6f2b2edb04e"
  },
  "commits": {
    "2024-08-17": "d58187752a6b1b924a68423e5cd029a2aadc8742",
    "2024-08-19": "49f3d8f8294d40a3c37a1e2ea3053be7c88f47cb",
    "2024-08-21": "53d77f7724490fa738fdb9ee36b66c501cbc3753",
    "2024-08-22": "5677f679d69afc0b0da4da710d9a404e6c3b42e5"
  }
}

~There can be duplicates at the moment like master and 2024-08-23. We should filter them out when processing commits.~

tonistiigi commented 2 months ago

SGTM, but maybe include .0 feature releases as well, so that progress between patch releases is tracked (eg. when we fix something in a patch release). For last release maybe include all patch releases.

Not critical as this can be done with custom refs, but any RC-s newer than latest feature release should also be included.

crazy-max commented 2 months ago

SGTM, but maybe include .0 feature releases as well, so that progress between patch releases is tracked (eg. when we fix something in a patch release). For last release maybe include all patch releases.

Not critical as this can be done with custom refs, but any RC-s newer than latest feature release should also be included.

Done to track zero releases as well. I will open an issue to track RC-s newer than latest feature.

thompson-shaun commented 2 months ago

Any case where we would want to back N commits but in the same day?

crazy-max commented 2 months ago

Any case where we would want to back N commits but in the same day?

I don't think there is a reason to have more than one. Maybe we could make priority on commit of the day being tagged but we already cover that for releases.