ericsnowcurrently
commented
1 year ago I would propose:
1) A public repo containing the actions and code 2) A private fork of (1) that actually connects to the self-hosted runners (and other orgs with their own machines would also have their own forks) 3) A separate public repo to store the results from (2) 4) Parameterizing a bunch of things in the actions that are currently hardcoded so the repositories used for (2) and (3) can easily be changed
That seems reasonable to me, assuming those concrete cases materialize. 😄
Currently, the Github Actions-based benchmarking infrastructure consists of the following:
1) A private repo (benchmarking) that contains the actions, some supporting Python code, as well as the raw and derived results. This repo needs to be private to secure the self-hosted runner machines. 2) The ideas repo where select results (mostly weekly and tagged CPython releases) are copied, for a high-level overview. (Side question: Do we still need this?) 3) A public mirror of the benchmarking repo (benchmarking-public) so that we can easily share results, as well as share the supporting infrastructure code.
Having both code and results in (1) made things easier to get started, but if others want to use the infrastructure code with their own self-hosted runners, it's a bit cumbersome that everything is together. Forks of the code certainly wouldn't want forks of the results.
I would propose:
1) A public repo containing the actions and code 2) A private fork of (1) that actually connects to the self-hosted runners (and other orgs with their own machines would also have their own forks) 3) A separate public repo to store the results from (2) 4) Parameterizing a bunch of things in the actions that are currently hardcoded so the repositories used for (2) and (3) can easily be changed
It's probably not worth doing this refactor unless there are some concrete cases of others wanting to reuse the actions and code, so this is just a placeholder for discussion for now. (@carljm expressed an interest in Discord).