search

payu-org / payu

A workflow management tool for numerical models on the NCI computing systems
Apache License 2.0
18 stars 25 forks source link

Handling of hidden input files and directories #377

Open micaeljtoliveira opened 8 months ago

micaeljtoliveira commented 8 months ago

We recently realised that payu was including the /g/data/ik11/inputs/JRA-55/RYF/v1-4/make_rhuss/.git directory and files therein in the input of a given run. That means it creates symlinks for all those files and also includes them in the manifest.

Although it has no effect on the calculation, the fact that those files are in the manifest is a bit annoying, as it makes it unnecessarily long and harder to read.

I was wondering if there was a legitimate case where a user would want to use hidden files/directories in a calculation. If not, them maybe payu should simply ignore those files. What do you think?

aidanheerdegen commented 8 months ago

Agreed hidden directories should be ignored.

Whether we should support recursive descent into input directories is a tricky one. It has come up in this issue where it occurs, but is not correctly implemented