search

LieberInstitute / SPEAQeasy

SPEAQeasy: portable LIBD RNA-seq pipeline using Nextflow. Check http://research.libd.org/SPEAQeasy-example/ for an example on how to use this pipeline and analyze the resulting output files.
http://lieberinstitute.github.io/SPEAQeasy
MIT License
6 stars 4 forks source link

Clean up output from generate_logs.sh #65

Open Nick-Eagles opened 3 years ago

Nick-Eagles commented 3 years ago

Currently, scripts/generate_logs.sh makes the assumption that all nextflow processes have completed (whether successfully or not). However, whenever the pipeline halts before completing (e.g. due to a temporary filesystem issue), some processes are typically killed mid-execution, leading to some files in some working directories being absent.

scripts/generate_logs.sh attempts to proceed, taking quite some time for larger experiments, and filling up the main log (`SPEAQeasy_output.log) with error messages such as:

cat: [insert work directory here]/.exitcode: No such file or directory

Instead, scripts/generate_logs.sh should properly handle missing files, possibly stating they are missing in the per-sample logs.

gpertea commented 1 year ago

I have just used a quick'n'clumsy patch for this in the jhpce_custom branch, see: https://github.com/LieberInstitute/SPEAQeasy/blob/jhpce_custom/scripts/generate_logs.sh .. maybe it could be fetched into master if you think it's acceptable.