search

ocurrent / current-bench

Experimental benchmarking infrastructure using OCurrent pipelines
Apache License 2.0
33 stars 17 forks source link

Support running benchmarks on multiple long-standing branches of a repository #430

Closed punchagan closed 1 year ago

punchagan commented 1 year ago

Previously, we only ran the benchmarks on the default branch of a repository, apart from the pull requests. This PR adds support to run benchmarks on multiple branches of a repository, not just the default branch.

NOTE: This also includes the change in #428. That PR can either be merged before this one, or simply closed, in favor of this.

Closes #426

punchagan commented 1 year ago

Overall the code seems good (bar my minor comments), but I am not able to properly test that it works. I tried adding

{ "name": "ElectreAAS/merlin",
  "branches": [ "master", "mock-without-label" ]
}

to environment/development.conf, and adding commits to my mock repo, but the only thing that seems to run is the master branch. What would be a better way to test it?

This is how I tested it, too. Did you restart the containers after changing the development.conf file? Also, we ignore commits which are older than 2 weeks old, when running benchmarks (See Pipeline.filter_stale_repositories). I can't think of any other reason why you're unable to test it.

punchagan commented 1 year ago

Overall the code seems good (bar my minor comments), but I am not able to properly test that it works. I tried adding

{ "name": "ElectreAAS/merlin",
  "branches": [ "master", "mock-without-label" ]
}

to environment/development.conf, and adding commits to my mock repo, but the only thing that seems to run is the master branch. What would be a better way to test it?

This is how I tested it, too. Did you restart the containers after changing the development.conf file? Also, we ignore commits which are older than 2 weeks old, when running benchmarks (See Pipeline.filter_stale_repositories). I can't think of any other reason why you're unable to test it.

EDIT: I see that your commits on the branch are just about 2 weeks old, but also I see there's a "Pending" status from your local current-bench deployment on the latest commit on mock-without-label. So, not sure what exactly happened here, but might be worth looking at the logs again?

ElectreAAS commented 1 year ago

Thanks for the verification. With more recent commits everything runs fine indeed! I'll consider this PR good to go when the other conversation is resolved :)

ElectreAAS commented 1 year ago

There was a CI failure that vanished when I rebuilt it, I have no clue what caused it. I'll consider this good to go.

punchagan commented 1 year ago

Thanks for the careful review and merge, @ElectreAAS !