Error in rule run_spades

[s-junguy]

Hi, I got an iostream error in spades because a file cannot be opened. I use Illumina paired-end reads and I had to restart the pipeline because of a time limit issue before so maybe that`s why it does not work...?

spades.log:

Exception caught Cannot open file '/home/s-junguy/atlas/runallspades/all/P19752-105/assembly/.bin_reads/paired_0.off': iostream error

== Error ==  system call for: "['/home/user/atlas/databases/conda_envs/f9b130b75c406b2ca1b149c2a4cd967f_/bin/spades-core', '/home/user/atlas/runallspades/all/P19752-105/assembly/K55/configs/config.info', '/home/user/atlas/runallspades/all/P19752-105/assembly/K55/configs/mda_mode.info', '/home/user/atlas/runallspades/all/P19752-105/assembly/K55/configs/meta_mode.info']" finished abnormally, OS return value: 4

[SilasK]

It is normal that assembly may take too long. But there are checkpoints for every step in atlas and even during the assembly.

You can try to delete the spades files in /home/s-junguy/atlas/runallspades/all/P19752-105/assembly/ folder and re-start. There is a readsfolder which ideally you shouldn't delete.

When you talk about time limit. You are working on a cluster system? isn't it? Try to use a cluster profile which allows atlas to submit each step as a job to the cluster. If the assembly still takes too long. It should use the checkpoints, and finally finish.

[s-junguy]

Thank you for your help! There are many files and directories in the assembly directory besides the reads directory. Should I delete them all before a restart?

[SilasK]

Yes, its only for one sample isn't it?

[SilasK]

Don't forget the hidden directories ..

[s-junguy]

Ich deleted everything in the assembly folder without the reads folder but I still got the same error. I do not understand why this file cannot be opened. I ran the pipeline on several samples but this is the only one that has this problem.

[SilasK]

There is nothing strange in the qc report for this sample?

Then delete the reads folder also. Atlas will do again the error correction, merging and assembly. Maybe there went something wrong before.

[SilasK]

For note there is also the iption --keep-going to continue with all the other samokes but eventually you want to solve this error.

[s-junguy]

Now that I deleted the whole assembly folder and restarted the pipeline with atlas run -w ~/atlas/runallspades/all -c ~/atlas/runallspades/config.yaml -j 256 all it starts with a different sample that has the same problem. I have 34 samples in the samples file and there are 11 samples where the assembly has been successfull. Now I have found 3 other samples with the same iostream error but when I deleted their assembly folder the pipeline starts with a different sample.

[s-junguy]

I deleted all the assembly folders from the samples where the assembly has not finished but when I restarted the pipeline I got the same iostream error from before. It seems like deleting the assembly folder is not the solution.

[SilasK]

Can you send me the reports/qcreport.html?

[s-junguy]

There is no folder report or a file qcreport.html

[SilasK]

Run first atlas run qc

[s-junguy]

The qc ran successfully. How can I skip QC now?

[SilasK]

It is skiped by default. What does the reposts in qcreports say?

[s-junguy]

QC_report.txt

I uploaded the QC report as a .txt file because .html is not supported here. I looked into it and I cannot see anything strange.

[SilasK]

Ok the qc reads are ok.

Maybe there is something during the error correction. When you see the error messages it should you show 3 or 4 input files.

You cna also see the latest log file in .snakemake/log/

Could you please run testreads.sh on each of the 4 input files. Maybe one of the files is empty or corrupt.

If you don't find anything strange. Probably best is then to submit an issue to the https://github.com/ablab/spades i don't know why there could be a problem. Sorry.

One option would be to update the spades version, maybe the error is fixed in the mean time

conda activate <conda env>
conda list spades # shows version
conda install spades=3.15.5
conda deactivate

the <conda env>you also find in the rule description in the log.

[s-junguy]

Thank you for your suggestions! I just cannot find the testreads.sh

[SilasK]

testreads.sh is a script of the bbmap package that should be installed together with atlas.

[s-junguy]

Thank you for your help! I reduced the number of threads and then it somehow worked.