Multiple tools fails with similar errors

[ehj000]

A user reported that several tool failed with similar errors. I have reproduced these errors and tested a few more tools. Something seems to be wrong on the system level. Here are a handful of tested tools and errors:

Sub-sample sequences files python: error while loading shared libraries: /usr/local/bin/../lib/libpython3.6m.so.1.0: cannot read file data: Input/output error

SPAdes python: error while loading shared libraries: /usr/local/bin/../lib/libpython3.6m.so.1.0: cannot read file data: Input/output error

Shovill FATAL: container creation failed: mount /proc/self/fd/7->/var/singularity/mnt/session/rootfs error: while mounting image /proc/self/fd/7: failed to mount squashfs filesystem: input/output error

seqtk_sample FATAL: container creation failed: mount /proc/self/fd/7->/var/singularity/mnt/session/rootfs error: while mounting image /proc/self/fd/7: failed to mount squashfs filesystem: input/output error

FASTQ to FASTA FATAL: could not open image /cvmfs/singularity.galaxyproject.org/all/galaxy_sequence_utils:1.1.5--py_0: image format not recognized

[bruggerk]

Think this is due to the cvmfs disk being full, I had a simliar problem running BWA.

I have cleared the cache on the slurm and head node, please let me know if the problem persists.

Steps done: sudo cvmfs_talk cleanup 0 sudo systemctl restart autofs

/k

[torfinnnome]

It's tempting to disable use of /cvmfs/singularity.galaxyproject.org/ until we figure this out.

[ehj000]

Some of the tools are now working, but others are still throwing (different) errors. Shovill: [kmc] Error: some error while reading fastq file, please contact authors [shovill] Hello galaxy [shovill] You ran: /usr/local/bin/shovill --outdir out --cpus 4 --ram 50 --R1 fastq_r1.fastqsanger.gz --R2 fastq_r2.fastqsanger.gz --namefmt contig%05d --dept

SPAdes: Unable to run this job due to a cluster error, please retry it later

[torfinnnome]

We are now (until we figure out what is wrong) not using pre-made containers from /cvmfs/singularity.galaxyproject.org/ on both production and test servers. Instead, containers are fetched on-demand from quay.io. Containers are cached.

Hopefully we will not see more "image format not recognized" errors.

@ehj000: These tools work on usegalaxy.eu with the same input data sets?

[ehj000]

Have tested the tool Shovill on .no .eu and .org, and it fails with the same error message all places. The tool runs fine with the same data on local installation of shovill...

[torfinnnome]

@ehj000 SPAdes: Fixed. Could you rerun your jobs, to verify it works?

[torfinnnome]

@ehj000 Shovill works for me, using a test data set (ecoli_1K_1.fq and ecoli_1K_2.fq)

Where can I find your input data, so I can debug a bit more?

[spades] ======= SPAdes pipeline finished.
[spades] 
[spades] SPAdes log can be found here: /srv/galaxy/jobs/005/5749/working/out/spades/spades.log
[spades] 
[spades] Thank you for using SPAdes!
[shovill] User supplied --nocorr, so not correcting contigs.
[shovill] Assembly is 999, estimated genome size was 958 (+4.28%)

[ehj000]

SPAdes works. Thanks. Have shared input data for testing of Shovill here: https://usegalaxy.no/u/bf52c02868d747c5b784466753839ed6/h/test-shovill

[torfinnnome]

I was able to run Shovill when I uncompressed your fastq.gz files:

[shovill] Assembly is 5034776, estimated genome size was 4364751 (+15.35%)
[shovill] Using genome graph file 'spades/assembly_graph_with_scaffolds.gfa' => 'contigs.gfa'
[shovill] Walltime used: 11 min 3 sec

[ehj000]

Yes, it runs with uncompressed fastqsanger type (the command line version accepts compressed input files). Thanks for resolving all the issues with the different tools.