would it be possible to add one QC step checking the integrity of fq.gz file before splitting them into chunks?
it seems the existence of incomplete fq.gz files will stuck the TAP pipeline.
ie. the abnormal file lingering in fqsplit step forever, and all other normal files are stuck before the merging bams step.
e.g. this fq.gz file is corrupt, and the TAP was neither stopping nor issuing errors, the whole pipeline is lingering on cluster...
would it be possible to add one QC step checking the integrity of fq.gz file before splitting them into chunks?
it seems the existence of incomplete fq.gz files will stuck the TAP pipeline. ie. the abnormal file lingering in fqsplit step forever, and all other normal files are stuck before the merging bams step.
e.g. this fq.gz file is corrupt, and the TAP was neither stopping nor issuing errors, the whole pipeline is lingering on cluster...
then I checked the fq.gz file and it shows:
[wang04@clust1-headnode bam2fq]$ zcat PGDX6865P1_WGS.sorted_processed.namesort.r1.fastq.gz | wc -l
gzip: PGDX6865P1_WGS.sorted_processed.namesort.r1.fastq.gz: unexpected end of file