parameterize-whether-a-demo-script-is-running-for-cicd-test

[david-thrower]

Background of the issue:

• We have showcase demo scripts for how to use Cerebros and demonstration of SOTA results.
• We want to use the same scripts for benchmarks in our CICD testing, but have a limit on resources for GitHub Actions.
• We don't want to maintain 2 separate scripts, one for showcase use and one for CICD tests.

Proposed solution:

• Parameterize whether the script it is running for CICD only.
• Option 1:
  • Create an environment variable in the Github runner like CICD_TEST, then make all the Python scripts that demonstrate how to use Cerebros, read in this variable, but default it to False if the variable does not exist.
  • If the execution environment the script runs in has the environment variable CICD_TEST set to true, then a small subset of the data is run in the training jobs. If the variable is set absent or set to false, then the full data set runs.
• Option 2:
  • Use fire.Fire or argparse to run the job, adding a shell arg: --CICD_TEST, which if set will trigger the small subset run.

[sashakolpakov]

We have one successful run, though: https://github.com/david-thrower/cerebros-core-algorithm-alpha/actions/runs/6721781884

On Wed, 1 Nov 2023 at 17:46, David Thrower @.***> wrote:

Background of the issue:

• We have showcase demo scripts for how to use Cerebros and demonstration of SOTA results.
• We want to use the same scripts for benchmarks in our CICD testing, but have a limit on resources for GitHub Actions.
• We don't want to maintain 2 separate scripts, one for showcase use and one for CICD tests.

Proposed solution:

• Parameterize whether the script it is running for CICD only.
• Option 1:
  • Create an environment variable in the Github runner like CICD_TEST, then make all the Python scripts that demonstrate how to use Cerebros, read in this variable, but default it to False if the variable does not exist.
  • If the execution environment the script runs in has the environment variable CICD_TEST set to true, then a small subset of the data is run in the training jobs. If the variable is set absent or set to false, then the full data set runs.
• Option 2:
  • Use fire.Fire or argparse to run the job, adding a shell arg: --CICD_TEST, which if set will trigger the small subset run.

— Reply to this email directly, view it on GitHub https://github.com/david-thrower/cerebros-core-algorithm-alpha/issues/124, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIOQZLEVH7CNFLZ73QJZKLYCJ4FFAVCNFSM6AAAAAA6ZSFWBGVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE3TENZSHA3TOOI . You are receiving this because you are subscribed to this thread.Message ID: @.***>