search

yyoshiaki / VIRTUS2

A bioinformatics pipeline for viral transcriptome detection and quantification considering splicing.
Other
16 stars 6 forks source link

poor support for Singularity #35

Open gailrosen opened 8 months ago

gailrosen commented 8 months ago

Hi,

to get singularity to run, I've had to go into almost all cwl scripts and put "#!/usr/bin/env -S cwltool --singularity" at the top.

Now, I STILL get a problem where it's trying to call Docker (i have put the cwltool --singularity at the top of fastp-pe.cwl but it still won't work.. I have tried everything.... even changing the alias of docker to singularity... why doesn't this work?!!):

INFO [workflow ] start INFO [workflow ] starting step fastp_pe INFO [step fastp_pe] start ERROR Workflow or tool uses unsupported feature: Docker is required to run this tool: docker executable is not available pigz: abort: write error on SRR11862683_1.fastq.gz (Inappropriate ioctl for device) pigz: abort: write error on SRR11862683_2.fastq.gz (Inappropriate ioctl for device)

2023-11-04T15:20:40 prefetch.3.0.7: Current preference is set to retrieve SRA Normalized Format files with full base quality scores. 2023-11-04T15:20:40 prefetch.3.0.7: 1) 'SRR11862684' is found locally 2023-11-04T15:20:40 prefetch.3.0.7: 'SRR11862684' has 0 unresolved dependencies spots read : 44,031,047 reads read : 88,062,094 reads written : 88,062,094 INFO /home/gailr/.local/bin/cwltool 3.1.20230906142556 INFO Resolved '/ifs/groups/rosenMRIGrp/gailr/VIRTUS/bin/VIRTUS.PE.cwl' to 'file:///ifs/groups/rosenMRIGrp/gailr/VIRTUS/bin/V IRTUS.PE.cwl' WARNING Workflow checker warning: ../../../bin/VIRTUS.PE.cwl:311:9: Source 'output' of type ["null", "File"] may be incompatible ../../../bin/VIRTUS.PE.cwl:257:9: with sink 'virus_bam' of type "File" ../../../bin/VIRTUS.PE.cwl:341:9: Source 'output' of type ["null", "File"] may be incompatible ../../../bin/VIRTUS.PE.cwl:348:9: with sink 'fq1' of type "File" ../../../bin/VIRTUS.PE.cwl:326:9: Source 'output' of type ["null", "File"] may be incompatible ../../../bin/VIRTUS.PE.cwl:350:9: with sink 'fq2' of type "File" ../../../bin/VIRTUS.PE.cwl:222:9: Source 'output_fq2' of type ["null", "File"] may be incompatible ../../../bin/VIRTUS.PE.cwl:321:9: with sink 'input_fq' of type "File" ../../../bin/VIRTUS.PE.cwl:185:9: Source 'mappingstats' of type ["null", "File"] may be incompatible ../../../bin/VIRTUS.PE.cwl:294:9: with sink 'input_STARLog' of type "File" INFO [workflow ] start INFO [workflow ] starting step fastp_pe INFO [step fastp_pe] start ERROR Workflow or tool uses unsupported feature: Docker is required to run this tool: docker executable is not available pigz: abort: write error on SRR11862684_1.fastq.gz (Inappropriate ioctl for device) pigz: abort: write error on SRR11862684_2.fastq.gz (Inappropriate ioctl for device)

gailrosen commented 8 months ago

[gailr@node006 batch_nima]$ which cwltool ~/.local/bin/cwltool [gailr@node006 batch_nima]$ ~/.local/bin/cwltool --version /home/gailr/.local/bin/cwltool 3.1.20230906142556 [gailr@node006 batch_nima]$ apptainer --version apptainer version 1.1.6-1.el8 [gailr@node006 batch_nima]$ singularity --version apptainer version 1.1.6-1.el8

gailrosen commented 8 months ago

i am also runnin the VIRTUS_wrapper.py

gailrosen commented 8 months ago

I don't get this error when I run VIRTUS.PE.cwl .... just when i run the wrapper. It seems to be in the way the commandline tool is run in CWL

gailrosen commented 8 months ago

I think it's working if I go into /home/gailr/.local/lib/python3.7/site-packages/cwltool/command_line_tool.py and manually set runtimeContext.singularity=True . And this is at the top of the make_job_runner function. If you can tell me how I was supposed to get this to work without completely hacking this, I would greatly appreciate it!

yyoshiaki commented 1 month ago

Hi, so sorry for my late reply. VIRTUS_wrappers.py have --singularity option. Did you try this option?

gailrosen commented 1 month ago

of course i did... but it still had probs

yyoshiaki commented 1 month ago

Hmm. If #!/usr/bin/env -S cwltool --singularity works, to reproduce this line, how about replacing cwltool in VIRTUS_wrapper.py to the full path to cwltool which can be checked by which cwltool though this is not essential solution? https://github.com/yyoshiaki/VIRTUS2/blob/master/wrapper/VIRTUS_wrapper.py#L126 https://github.com/yyoshiaki/VIRTUS2/blob/master/wrapper/VIRTUS_wrapper.py#L140

Because this looks like the error relating to cwltool itself, could you ask for it in the cwltool repository if the error persists? You may get better information there.