search

yyoshiaki / VIRTUS

A bioinformatics pipeline for viral transcriptome detection and quantification considering splicing.
https://github.com/yyoshiaki/VIRTUS/wiki
Other
34 stars 9 forks source link

EXITING because of FATAL ERROR: could not create output file: ...virtus143/Log.out Check if the path ... exists and you have permissions to write there #33

Open ozy913 opened 1 year ago

ozy913 commented 1 year ago

Hi Yoshiaki, While I was trying to run the VIRTUS.PE.cwl, it throw an error "EXITING because of FATAL ERROR: could not create output file: /media/ozy/Seagate/virtus143/Log.out Check if the path /media/ozy/Seagate/virtus143/ exists and you have permissions to write there". Intrestingly, test.sh runs successfully. Moreover, it passes the fastp step. But later the star and salmon can not initiate. Do you have any suggestion for this error?

Code: ozy@ozy:/media/ozy/Seagate/virtus143$ sudo cwltool --tmp-outdir-prefix=/media/ozy/Seagate/virtus_tmp/ --tmpdir-prefix=/media/ozy/Seagate/virtus_tmp/ ~/VIRTUS/workflow/VIRTUS.PE.cwl --fastq2 /media/ozy/Seagate/fastq/SRR1220143_2.fastq.gz --fastq1 /media/ozy/Seagate/fastq/SRR1220143_1.fastq.gz --genomeDir_human /home/ozy/VIRTUS/test/STAR_index_human --outFileNamePrefix_human /media/ozy/Seagate/virtus143/ --nthreads 10 --genomeDir_virus /home/ozy/VIRTUS/test/STAR_index_virus --salmon_index_human /home/ozy/VIRTUS/test/salmon_index_human --salmon_quantdir_human /media/ozy/Seagate/virtus143/salmon_human/ Error: [sudo] password for ozy: INFO /usr/bin/cwltool 3.1.20220224085855 INFO Resolved '/home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl' to 'file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl' WARNING Workflow checker warning: ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:222:9: Source 'output_fq2' of type ["null", "File"] may be incompatible ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:321:9: with sink 'input_fq' of type "File" ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:341:9: Source 'output' of type ["null", "File"] may be incompatible ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:348:9: with sink 'fq1' of type "File" ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:326:9: Source 'output' of type ["null", "File"] may be incompatible ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:350:9: with sink 'fq2' of type "File" ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:311:9: Source 'output' of type ["null", "File"] may be incompatible ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:257:9: with sink 'virus_bam' of type "File" ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:185:9: Source 'mappingstats' of type ["null", "File"] may be incompatible ../../../../home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl:294:9: with sink 'input_STARLog' of type "File" INFO [workflow ] start INFO [workflow ] starting step fastp_pe INFO [step fastp_pe] start INFO [job fastp_pe] /media/ozy/Seagate/virtus_tmp/6xg2grpd$ docker \ run \ -i \ --mount=type=bind,source=/media/ozy/Seagate/virtus_tmp/6xg2grpd,target=/uiSllf \ --mount=type=bind,source=/media/ozy/Seagate/virtus_tmp/2dnb7yop,target=/tmp \ --mount=type=bind,source=/media/ozy/Seagate/fastq/SRR1220143_1.fastq.gz,target=/var/lib/cwl/stgf37e20a1-721f-4ea7-86ec-502b5615b8fe/SRR1220143_1.fastq.gz,readonly \ --mount=type=bind,source=/media/ozy/Seagate/fastq/SRR1220143_2.fastq.gz,target=/var/lib/cwl/stgd7c96872-e1ac-4a18-8e70-0ea75e1d947f/SRR1220143_2.fastq.gz,readonly \ --workdir=/uiSllf \ --read-only=true \ --user=0:0 \ --rm \ --cidfile=/media/ozy/Seagate/virtus_tmp/tsbad779/20230325211152-722335.cid \ --env=TMPDIR=/tmp \ --env=HOME=/uiSllf \ quay.io/biocontainers/fastp:0.20.0--hdbcaa40_0 \ /bin/sh \ -c \ 'fastp' '-o' 'SRR1220143_1.fastp.fastq' '-O' 'SRR1220143_2.fastp.fastq' --trim_poly_x '-i' '/var/lib/cwl/stgf37e20a1-721f-4ea7-86ec-502b5615b8fe/SRR1220143_1.fastq.gz' '-I' '/var/lib/cwl/stgd7c96872-e1ac-4a18-8e70-0ea75e1d947f/SRR1220143_2.fastq.gz' '--length_required' '40' '--thread' '10' Read1 before filtering: total reads: 49340608 total bases: 4440654720 Q20 bases: 4351342389(97.9888%) Q30 bases: 4111905237(92.5968%)

Read2 before filtering: total reads: 49340608 total bases: 4440654720 Q20 bases: 4342603141(97.792%) Q30 bases: 4099225180(92.3113%)

Read1 after filtering: total reads: 49077128 total bases: 4415043920 Q20 bases: 4332747257(98.136%) Q30 bases: 4097754668(92.8135%)

Read2 aftering filtering: total reads: 49077128 total bases: 4415040603 Q20 bases: 4325663710(97.9756%) Q30 bases: 4086692407(92.563%)

Filtering result: reads passed filter: 98154256 reads failed due to low quality: 521308 reads failed due to too many N: 0 reads failed due to too short: 5652 reads with adapter trimmed: 149318 bases trimmed due to adapters: 1075926 reads with polyX in 3' end: 286150 bases trimmed in polyX tail: 3047781

Duplication rate: 11.5091%

Insert size peak (evaluated by paired-end reads): 136

JSON report: fastp.json HTML report: fastp.html

fastp -o SRR1220143_1.fastp.fastq -O SRR1220143_2.fastp.fastq --trim_poly_x -i /var/lib/cwl/stgf37e20a1-721f-4ea7-86ec-502b5615b8fe/SRR1220143_1.fastq.gz -I /var/lib/cwl/stgd7c96872-e1ac-4a18-8e70-0ea75e1d947f/SRR1220143_2.fastq.gz --length_required 40 --thread 10 fastp v0.20.0, time used: 371 seconds INFO [job fastp_pe] Max memory used: 3081MiB INFO [job fastp_pe] completed success INFO [step fastp_pe] completed success INFO [workflow ] starting step star_mapping_pe_human INFO [step star_mapping_pe_human] start INFO [job star_mapping_pe_human] /media/ozy/Seagate/virtus_tmp/2pf05556$ docker \ run \ -i \ --mount=type=bind,source=/media/ozy/Seagate/virtus_tmp/2pf05556,target=/uiSllf \ --mount=type=bind,source=/media/ozy/Seagate/virtus_tmp/3er0w3f2,target=/tmp \ --mount=type=bind,source=/media/ozy/Seagate/virtus_tmp/6xg2grpd/SRR1220143_1.fastp.fastq,target=/var/lib/cwl/stgdf62fff5-1a7e-4635-bd35-ba99796f66e3/SRR1220143_1.fastp.fastq,readonly \ --mount=type=bind,source=/media/ozy/Seagate/virtus_tmp/6xg2grpd/SRR1220143_2.fastp.fastq,target=/var/lib/cwl/stgea691e76-23e6-48a6-b597-7355cecd0db2/SRR1220143_2.fastp.fastq,readonly \ --workdir=/uiSllf \ --read-only=true \ --user=0:0 \ --rm \ --cidfile=/media/ozy/Seagate/virtus_tmp/f88fekif/20230325212500-521126.cid \ --env=TMPDIR=/tmp \ --env=HOME=/uiSllf \ quay.io/biocontainers/star:2.7.1a--0 \ /bin/sh \ -c \ 'STAR' '--runMode' 'alignReads' --genomeDir /uiSllf/STAR_index_human --runThreadN 10 --outFileNamePrefix /media/ozy/Seagate/virtus143/ --outSAMtype BAM SortedByCoordinate --outSAMunmapped Within '--readFilesIn' '/var/lib/cwl/stgdf62fff5-1a7e-4635-bd35-ba99796f66e3/SRR1220143_1.fastp.fastq' '/var/lib/cwl/stgea691e76-23e6-48a6-b597-7355cecd0db2/SRR1220143_2.fastp.fastq'

EXITING because of FATAL ERROR: could not create output file: /media/ozy/Seagate/virtus143/Log.out Check if the path /media/ozy/Seagate/virtus143/ exists and you have permissions to write there

Mar 25 18:25:00 ...... FATAL ERROR, exiting INFO [job star_mapping_pe_human] Max memory used: 0MiB WARNING [job star_mapping_pe_human] exited with status: 102 ERROR [job star_mapping_pe_human] Job error: ("Error collecting output for parameter 'aligned': ../../../../home/ozy/VIRTUS/tool/star/star_mapping-pe/star_mapping-pe.cwl:83:7: glob patterns must not start with '/'", {}) WARNING [job star_mapping_pe_human] completed permanentFail ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/aligned ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/bamRemDups ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/mappingstats ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/readspergene ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/transcriptomesam ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/Log.out ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/Log.progress.out ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/SJ.out.tab ERROR [step star_mapping_pe_human] Output is missing expected field file:///home/ozy/VIRTUS/workflow/VIRTUS.PE.cwl#VIRTUS.PE/star_mapping_pe_human/unmapped WARNING [step star_mapping_pe_human] completed permanentFail INFO [workflow ] completed permanentFail { "Log.out_human": null, "Log.progress.out_human": null, "SJ.out.tab_human": null, "aligned_bam_human": null, "output_unmapped": null, "output_fq2": null, "output_fq1": null, "SJ.out.tab_virus": null, "Log.progress.out_virus": null, "Log.out_virus": null, "aligned_virus": null, "mappingstats_virus": null, "mappingstats_human": null, "output_quantdir_human": null, "output": null, "output_1": null } WARNING Final process status is permanentFail ozy@ozy:/media/ozy/Seagate/virtus143$ sudo cwltool --rm-tmpdir ~/VIRTUS/workflow/VIRTUS.PE.cwl --fastq2 /media/ozy/Seagate/fastq/SRR1220143_2.fastq.gz --fastq1 /media/ozy/Seagate/fastq/SRR1220143_1.fastq.gz --genomeDir_human /home/ozy/VIRTUS/test/STAR_index_human --outFileNamePrefix_human /media/ozy/Seagate/virtus143/ --nthreads 8 --genomeDir_virus /home/ozy/VIRTUS/test/STAR_index_virus --salmon_index_human

patrick-douglas commented 1 year ago

Dear @ozy913 Are you running as a default user? Try running VIRTUS as root instead!

Docker usually works better as root.

Best regards Patrick~