User-scripts error-handling, commands and tests

[jo-basevi]

• Raise RuntimeErrors when user-scripts exit with an error (previously it was just a warning)
• Move run user-script code to payu.fsops out of Experiment class so it's easier to test
• Use shell=True for subprocess commands needing a shell, e.g. file redirections, pipes
• Add tests for running userscripts

To be able to run commands such as echo "some_data" > input.txt that are in payu documentation, it requires use of shell=True. So there's some security issues with using shell=True (https://docs.python.org/3/library/subprocess.html#security-considerations). However as the user scripts have typically been user defined .sh files run with /bin/bash, it could contain malicious code anyway..

I added a needs_subprocess_shell function that checks for any shell specific values. The list I've added might not be exhaustive. The question I am not sure on is that function even needed, and should all the user script commands just be run with shell=True?

Closes #448, closes #444

[pep8speaks]

Hello @jo-basevi! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2024-07-01 01:05:59 UTC