We have a data management pipeline to first tar the data and then archive it.
I need to check that the data is correct (not corrupted during tar process) before
archive. And All the data management pipeline is executed through script (
submit dtar job to slurm(pbs) and archive). I need some way to automatically
detect if dtar has success.
The doc says:
One cannot easily check the return codes of tools. Instead, inspect stdout and stderr output for errors.
maybe I need to use regular expression to match the ouput of dtar?(maybe slurm.out or something)?
Thanks
We have a data management pipeline to first tar the data and then archive it. I need to check that the data is correct (not corrupted during tar process) before archive. And All the data management pipeline is executed through script ( submit dtar job to slurm(pbs) and archive). I need some way to automatically detect if dtar has success.
The doc says:
maybe I need to use regular expression to match the ouput of dtar?(maybe slurm.out or something)? Thanks