problem with tmp folder

[Sebdu132]

Dear Malte,

I am part of the FULLFORCE project, and I'm facing problems with FFPA.

I am planning to run it on a local machine to make some tests, but I have an error at the beginning of the execution. Docker and the containers are properly installed, and I am running FFPA in my home directory. Here is the command line:

python3 ./FFPA.py -i_trimmed_illumina /home/minion/Bureau/FULLFORCE/TestData/13_R1.fastq /home/minion/Bureau/FULLFORCE/TestData/13_R2.fastq -i_raw_nanopore /home/minion/Bureau/FULLFORCE/TestData/nanopore/nanopore_13.fastq -trimmomatic_db "Nextera" -nanoporeqscore 8 -o ASSEMBLY

And here is the first part of the execution with errors from the very beginning, with makes the following treaments not relevant:

gzip: compressed data not read from a terminal. Use -f to force decompression. For help, type: gzip -h Trimmomatic started File provided doesn't exist or the path is incorrect: /tmp/nanopore_13.fastq c9b3fc78ad48 Skipping '/tmp/input/13_R1.fastq' which didn't exist, or couldn't be read Skipping '/tmp/input/13_R2.fastq' which didn't exist, or couldn't be read a223a01aa929 a03aa5e0239c nanofilt q complete 31ad516a7896 nanofilt l 10000 complete kraken: can't determine what format /tmp/input/nanopore_13.fastq.q8_nanofilt.fastq is! Error: No such container:path: 5e61b7346596:/tmp/krakenoutput_nanopore 5e61b7346596 kraken finished nanopore q 86ea2ab3d155 13_R1.fastq 13_R2.fastq docker run --name kraken_container59399 -it -v /home/minion/Bureau/FULLFORCE/FullForcePlasmidAssembler/ASSEMBLY/tmp/illuminaPE_trimmed/:/tmp/input/ flowcraft/kraken:1.0-0.1 kraken --db /kraken_db/minikraken_20171013_4GB --output /tmp/krakenoutput_illumina /tmp/input/13_R1.fastq /tmp/input/13_R2.fastq classify: can't open /tmp/input/13_R1.fastq: No such file or directory 8e5b86143a1e kraken finished illumina PE 95191ebb03dc rm: impossible de supprimer '/home/minion/Bureau/FULLFORCE/FullForcePlasmidAssembler/ASSEMBLY//tmp/krakenoutput_nanopore': est un dossier

So for what I can understand is that FFPA create a "tmp" directory into the output directory (=/home/minion/Bureau/FULLFORCE/FullForcePlasmidAssembler/ASSEMBLY) but then it tries to search for the files in a rooted tmp directory "/tmp/nanopore_13.fastq" and others.

According to the other prints, it seems that sometimes it wants to use /tmp and sometimes $OUTPUTPATH/tmp... How can I handle this?

Sebastien.

[MBHallgren]

Hi Sebastien,

Should have been fixed now, and both gzipped and unzipped input data can now be used.

Best regards, Malte

[Sebdu132]

Hi Malte,

thanks for the fix but unfortunately it doesn't handle all errors. When I run : python3 ./FFPA.py -i_trimmed_illumina /home/minion/Bureau/FULLFORCE/TestData/13_R1.fastq /home/minion/Bureau/FULLFORCE/TestData/13_R2.fastq -i_raw_nanopore /home/minion/Bureau/FULLFORCE/TestData/nanopore/nanopore_13.fastq -trimmomatic_db "Nextera" -nanoporeqscore 8 -o ASSEMBLY

I still have issues:

Trimmomatic started
Qcat is running, be patient :) 
[Errno 21] Is a directory: '/tmp/nanopore_13.fastq.fastq'
qcat complete
Error: No such container:path: 86317ac9326a:/tmp/nanopore_13.fastq_trimmed.fastq
86317ac9326a
File provided doesn't exist or the path is incorrect: /tmp/nanopore_13.fastq_trimmed.fastq
c11eb1e0a0be
Skipping '/tmp/input/13_R1.fastq' which didn't exist, or couldn't be read
Skipping '/tmp/input/13_R2.fastq' which didn't exist, or couldn't be read
d9684cf6e4b9
33acfd675161
nanofilt q complete
729bd2aa1350
nanofilt l 10000 complete
kraken: can't determine what format /tmp/input/nanopore_13.fastq.q8_nanofilt.fastq is!
Error: No such container:path: e05027c073d8:/tmp/krakenoutput_nanopore
e05027c073d8
kraken finished nanopore q
...

And indeed the nanopore file is created as a directory ! :

minion@TRS-48PTM62:~/Bureau/FULLFORCE/FullForcePlasmidAssembler$ ll ASSEMBLY/tmp/
total 926460
drwxrwxr-x 11 minion minion      4096 nov.  17 10:00 ./
drwxrwxr-x  4 minion minion      4096 nov.  17 09:59 ../
drwxrwxrwt  4 minion minion      4096 nov.  17 09:59 fastqc/
drwxrwxr-x  2 minion minion      4096 nov.  17 10:00 hybridinput/
drwxrwxr-x  2 minion minion      4096 nov.  17 09:59 illuminaPE/
drwxr-xr-x  2 root   root        4096 nov.  17 09:59 illuminaPE_trimmed/
drwxr-xr-x  2 root   root        4096 nov.  17 09:59 krakenoutput_nanopore/
-rw-rw-r--  1 minion minion        29 nov.  17 10:00 kraken_report_illumina_1percenthits
-rw-rw-r--  1 minion minion        45 nov.  17 10:00 kraken_report_nanopore_1percenthits
drwxr-xr-x  2 minion minion      4096 nov.  17 09:59 nanoplots/
-rw-rw-r--  1 minion minion 948626113 nov.  17 09:59 nanopore_13.fastq
-rw-rw-r--  1 minion minion       410 nov.  17 09:59 nanopore_13.fastq.10000.nanofilt
drwxr-xr-x  2 root   root        4096 nov.  17 09:59 nanopore_13.fastq.fastq/
-rw-rw-r--  1 minion minion       410 nov.  17 09:59 nanopore_13.fastq.q8_nanofilt.fastq
drwxr-xr-x  2 root   root        4096 nov.  17 09:59 nanopore_13.fastq_trimmed.fastq/
drwxr-xr-x  2 minion minion      4096 nov.  17 10:00 nanopore_assembly/

I will try to correct this on my own if the error comes from the FFPA.py code, but it may also come from one of the embedded docker code, is such case I can't manage it.

regards, Sebastien.

[MBHallgren]

Hi Sebastien,

It appears the file was stilll being stored incorrectly, and the embedded docker image automatically tries to create a directory if it can't find the file. I think it has been fixed now - I just ran the whole pipeline without any problems locally here at least.

Let me know, if it is working for you now.

best regards, Malte