Open dougiesquire opened 4 months ago
Sorry to equivocate, but all the options you've presented sound reasonable, depending on the specifics of the use case.
To my mind setup
and check
are close to the same thing. The setup
option would have to call a check
routine, or something similar. Creating check
as a CLI argument would just be a case of making that internal command callable separately.
The question of setup
or userscript
probably comes down to how specific are these checks to a specific configuration? If they are something that would be applicable in all cases for a model, then adding it to the model driver makes sense. If the checks are more configuration specific, if there are cases where the checks might not be necessary, then userscript
makes more sense.
The downside of userscript
is you don't have access to all the internal payu
information. If this was necessary there might be a case for a check
routine, which isn't called by default, but could be specified as a userscript
option for specific configs.
Perhaps some specific examples would help to elucidate this?
As an extension I'd like to see checks like this be part of CI when updating released configurations.
To my mind setup and check are close to the same thing. The setup option would have to call a check routine, or something similar. Creating check as a CLI argument would just be a case of making that internal command callable separately.
Agreed. I just wondered whether it might be cleanest to have the checks in their own method. It would make it clear to new driver developers, for example, that checks are a good idea and where they should live. But yes, it boils down to the same thing, and thinking about it more I can't see any value in having separate setup
and check
subcommands.
All of the checks I want to run at the moment make the most sense in the (access-om3) driver. But, I wanted to check there were no strong opinions about whether/where/how they should be included in Payu.
As an extension I'd like to see checks like this be part of CI when updating released configurations.
Yes! And if the checks are in Payu we could use Payu to do this
All of the checks I want to run at the moment make the most sense in the (access-om3) driver. But, I wanted to check there were no strong opinions about whether/where/how they should be included in Payu.
If these are checks that one would always want to run for all versions of access-om3
then yeah, sounds like the driver is the right place for them.
Agreed a checks
method is clean, and can be exposed as a CLI at a later date, or added as a --checks
option to setup
if it isn't something that should always be run (which it may be).
We're implementing some QA checks in CI for the access-om2 configurations
https://github.com/ACCESS-NRI/access-om2-configs/pull/32
They're an example of checks we want to do for released configs, but not routinely when running the model. Just FYI.
Often it is important that different parameters in a configuration are set consistently. These may be different but related parameters, or one parameter that needs to be set to the same value in multiple places (yuck, I know, but we're stuck with this unfortunately). There's an argument for including simple checks like these in Payu, to be run automatically as part of
payu run
. If you agree, where's the right place to do this:setup
step of a model driver (there are some checks in some drivers already)?userscript
?check
method (and subcommand)?