jpuritz / dDocent

a bash pipeline for RAD sequencing
ddocent.com
MIT License
52 stars 42 forks source link

Fastp not getting number of processors #59

Closed kubu4 closed 4 years ago

kubu4 commented 4 years ago

When trying to run dDocent with a config file, we're getting the following error:

mawk: cannot open uniq.fq1 (No such file or directory)

This seems to point to this section of dDocent:

fastp -i uniq.fq -o uniq.fq1 -w $NP -Q &> assemble.trim.log
mawk 'BEGIN{P=1}{if(P==1||P==2){gsub(/^[@]/,">");print}; if(P==4)P=0; P++}' uniq.fq1 | paste - - | sort -k1,1 -V | tr "\t" "\n" > uniq.fasta
mawk '!/>/' uniq.fasta > totaluniqseq
rm uniq.fq*

When we look at the assemble.trim.log file, we can see that the value for $NP is not getting passed to the -w option:

head assemble.trim.log

head logfiles/assemble.trim.log 
option value is invalid: --thread=-Q
usage: fastp [options] ... 
options:
  -i, --in1                            read1 input file name (string [=])
  -o, --out1                           read1 output file name (string [=])
  -I, --in2                            read2 input file name (string [=])
  -O, --out2                           read2 output file name (string [=])
      --unpaired1                      for PE input, if read1 passed QC but read2 not, it will be written to unpaired1. Default is to discard it. (string [=])
      --unpaired2                      for PE input, if read2 passed QC but read1 not, it will be written to unpaired2. If --unpaired2 is same as --umpaired1 (default mode), both unpaired reads will be written to this same file. (string [=])
      --failed_out                     specify the file to store reads that cannot pass the filters. (string [=])

We confirmed that $NUMProc is getting the correct number of processors that it pulls from the config file:

head dDocent.runs

Variables used in dDocent Run at Tue Nov 26 04:38:06 PST 2019
Number of Processors
8
Maximum Memory
0
Trimming
yes
Assembly?
yes
Type_of_Assembly

Any idea what's happening and/or suggestions on how to fix it?

We haven't attempted to "hard code" the number of processors in the fastp -w option, and would like to avoid doing so.

Thanks for any insight.

jpuritz commented 4 years ago

Which version are you using. This was corrected a few version ago.

kubu4 commented 4 years ago

v2.7.6.

This is the version linked on the dDocent downloads page: http://www.ddocent.com/downloads/

jpuritz commented 4 years ago

Link on ddocent.com is now updated to v2.7.8 which is the latest official release (also available on bioconda).