Command sent to cluster only partially executed [rnaseqqc]

[jscaber]

@AndreasHeger

Hi Andreas,

This but was discovered in rnaseqqc but may have wider implications. When presented with two fastq files, the function subsetSequenceData generates only one subset: File1.fastq.1.gz File1.subset etc.

The command that is sent to the cluster is correct. It can be executed on the command line, where it runs fine. We have tested this extensively with Sebastian on both mine and his set-ups and have not found the cause.

2018-09-12 10:40:02,415 INFO main Execution - {"task": "pipeline_rnaseqqc.subsetSequenceData", "statement": "mkdir -p /ifs/projects/proj096/rnaseqqc/ctmpkhn62bg3; zcat SMA-0205-1.fastq.1.gz                     | aw
k 'NR > 4000000 {exit} {print}'                     | gzip                     > fastq.dir/SMA-0205-1.fastq.1.gz; zcat SMA-0205-1.fastq.2.gz                     | awk 'NR > 4000000 {exit} {print}'                  
   | gzip                     > fastq.dir/SMA-0205-1.fastq.2.gz;  rm -rf /ifs/projects/proj096/rnaseqqc/ctmpkhn62bg3", "hostname": "cgat016.anat.ox.ac.uk", "job_id": "0", "engine": "GridExecutor", "submit_time": 15
36745168.1428206, "start_time": 1536745168.1428206, "end_time": 1536745202.4140775, "slots": 1.0, "exit_status": 0.0, "total_t": 34.27125692367554, "cpu_t": 0.0, "wall_t": 0.0, "user_t": 0.0, "sys_t": 0.0, "child_u
ser_t": 0.0, "child_sys_t": 0.0, "shared_data": 0.0, "io_input": 0.0, "io_output": 0.0, "average_memory_total": 0.0, "percent_cpu": 0.0, "average_rss": 0.0, "max_rss": 0.0, "max_vmem": 0.0, "minor_page_faults": 0.0
, "swapped": 0.0, "context_switches_involuntarily": 0.0, "context_switches_voluntarily": 0.0, "average_uss": 0.0, "signal": 0.0, "socket_received": 0.0, "socket_sent": 0.0, "major_page_faults": 0.0, "unshared_data"
: 0.0}

The code after the second semi-colon does not appear to be run. There are no error messages.

Best wishes, Jakub

[jscaber]

PS: When reverting back to the CGATOxford code, the task runs fine.

[jscaber]

I have rewritten the code so it now inserts && instead of ; to join statments (#52), but this does not work either (cannot change the ";" before and after tempdir):

# 2018-09-12 11:21:57,245 INFO {"task": "pipeline_rnaseqqc.subsetSequenceData", "statement": "mkdir -p /ifs/projects/proj096/rnaseqqc/ctmpm5qc31vq; zcat SMA-0303-3.fastq.1.gz                     | awk 'NR > 4000000 {exit} {print}'                     | gzip                     > fastq.dir/SMA-0303-3.fastq.1.gz && zcat SMA-0303-3.fastq.2.gz                     | awk 'NR > 4000000 {exit} {print}'                     | gzip                     > fastq.dir/SMA-0303-3.fastq.2.gz ;  rm -rf /ifs/projects/proj096/rnaseqqc/ctmpm5qc31vq", "hostname": "cgat014.anat.ox.ac.uk", "job_id": "0", "engine": "GridExecutor", "submit_time": 1536747675.1106207, "start_time": 1536747675.1106207, "end_time": 1536747717.2442586, "slots": 1.0, "exit_status": 0.0, "total_t": 42.13363790512085, "cpu_t": 0.0, "wall_t": 0.0, "user_t": 0.0, "sys_t": 0.0, "child_user_t": 0.0, "child_sys_t": 0.0, "shared_data": 0.0, "io_input": 0.0, "io_output": 0.0, "average_memory_total": 0.0, "percent_cpu": 0.0, "average_rss": 0.0, "max_rss": 0.0, "max_vmem": 0.0, "minor_page_faults": 0.0, "swapped": 0.0, "context_switches_involuntarily": 0.0, "context_switches_voluntarily": 0.0, "average_uss": 0.0, "signal": 0.0, "socket_received": 0.0, "socket_sent": 0.0, "major_page_faults": 0.0, "unshared_data": 0.0}

[AndreasHeger]

Thanks, I will check.

[AndreasHeger]

Hi Jakub, when I look in the test pipelines, both files are created:

-rw-rw-r--  1 andreas usersfgu 21776276 Sep  7 21:33 Brain-F1-R1.fastq.gz
-rw-rw-r--  1 andreas usersfgu        0 Sep  7 21:33 Brain-F1-R1.subset
-rw-rw-r--  1 andreas usersfgu 21846402 Sep  7 21:33 Brain-F1-R2.fastq.gz
-rw-rw-r--  1 andreas usersfgu        0 Sep  7 21:33 Brain-F1-R2.subset
-rw-rw-r--  1 andreas usersfgu 13760136 Sep  7 21:33 Brain-F2-R1.fastq.gz
-rw-rw-r--  1 andreas usersfgu        0 Sep  7 21:33 Brain-F2-R1.subset
-rw-rw-r--  1 andreas usersfgu 17543563 Sep  7 21:33 Brain-F2-R2.fastq.gz
-rw-rw-r--  1 andreas usersfgu        0 Sep  7 21:33 Brain-F2-R2.subset

which directory/project are you working in?

[AndreasHeger]

Never mind, I see it in the logs. Let me try.

[AndreasHeger]

I can reproduce it, thanks!

[AndreasHeger]

The reason is likely the interrupted pipe to gzip. This causes the whole statement to fail. It passes in the tests, because the number of reads is below the subsetting threshold.

[AndreasHeger]

... though an error code is not generated.

[AndreasHeger]

and ignore_pipe_errors was set to True.

[AndreasHeger]

... actually, it traces to a type in pipeline_rnaseqqc.py:

ignore_pipe_erors = True