hpc / mpifileutils

File utilities designed for scalability and performance.
https://hpc.github.io/mpifileutils
BSD 3-Clause "New" or "Revised" License
170 stars 68 forks source link

How to check if dtar succesfully finished? #577

Open yr-wang-hit opened 4 months ago

yr-wang-hit commented 4 months ago

We have a data management pipeline to first tar the data and then archive it. I need to check that the data is correct (not corrupted during tar process) before archive. And All the data management pipeline is executed through script ( submit dtar job to slurm(pbs) and archive). I need some way to automatically detect if dtar has success.

The doc says:

One cannot easily check the return codes of tools. Instead, inspect stdout and stderr output for errors.

maybe I need to use regular expression to match the ouput of dtar?(maybe slurm.out or something)? Thanks