Should `benchcab` have only one main program entry point?

[SeanBryan51]

Currently, benchcab uses two main program entry points: benchcab and benchsiterun.

• benchcab does the setup to run all the different test configurations
• benchsiterun runs CABLE over the different test configurations and is invoked via the job script (which is submitted by benchcab)

Problem:

• ideally, benchsiterun is invoked only by benchcab through the job script where the necessary setup steps have been completed (for example creating the directory structures, checking out and building CABLE, etc). However, benchsiterun is accessible to the user at the same level as benchcab which may give the user the impression that it is an additional tool to benchcab. This forces us to include a number of checks in benchsiterun.py as we cannot trust the user to use benchsiterun correctly (for example checking the directory structure is valid, relevant input files and executables exist, etc).

Is there a better solution where benchsiterun can only be invoked internally by benchcab and not by the user?

A possible solution could be:

1. Adding a flag to benchcab that disables the job script submission and instead lets users run CABLE on the same compute node they happen to run benchcab from.
2. Implement the ability to run specific steps of benchcab in isolation. For example, the checkout step, the build step, and the run step.

In the job script, instead of running benchsiterun, we can run benchcab with a flag that specifies the run step to be executed (preventing any the job script related code from being executed).

[ccarouge]

I see the issue here. I agree that having benchsiterun as a cli isn't ideal. It isn't meant to be run independently. A solution using a flag seems logical to me.