search

payu-org / payu

A workflow management tool for numerical models on the NCI computing systems
Apache License 2.0
18 stars 25 forks source link

Default to reproduce executable manifests hashes #383

Open aidanheerdegen opened 7 months ago

aidanheerdegen commented 7 months ago

payu has the ability to require reproducibility of executables, inputs and restarts by checking manifest hashes match.

The ability to specify reproducibility separately for each of these was introduced in https://github.com/payu-org/payu/issues/203 but the default was False for all three.

There is a suggestion to make manifest/exe.yaml reproduce default to true. See https://github.com/payu-org/payu/issues/379#issuecomment-1807259497 for details.

This makes it safer, by default, for users who don't ever intend to compile their own model executables. By default they are ensuring the executables they're using are the same as those listed in the exe manifest.

This can be circumvented by setting reproduce False in config.yaml.

(Note that just deleting manifest/exe.yaml doesn't work to circumvent this check. payu reports an error because there is no hash to compare against)

The biggest drawback to making this change would be for users who are compiling their own executables, or changing executables to other versions. With the code as it is this would require them to set this in config.yaml:

manifest:
   reproduce:
      exe: False

which is slightly annoying, and would then lead to the situation where they would have to either change it back to True or delete it to enable the exe reproduce checking again.

If it was decided to make this change then I would advocate for a command line option to override it, to make generating new exe manifests easier, but also require active input from the user.

aidanheerdegen commented 7 months ago

Note the relevant code changes would have to be made here:

https://github.com/payu-org/payu/blob/master/payu/manifest.py#L277-L281

micaeljtoliveira commented 7 months ago

I think this would be a good change.

If it was decided to make this change then I would advocate for a command line option to override it, to make generating new exe manifests easier, but also require active input from the user.

Fully agree with this.

aekiss commented 5 months ago

I like the idea of defaulting to reproducible exe, with CLI override