Random Genome Failings - Githubissues

oschwengers / asap

A scalable bacterial genome assembly, annotation and analysis pipeline

https://doi.org/10.1371/journal.pcbi.1007134

GNU General Public License v3.0

66 stars 18 forks source link

Random Genome Failings #20

Closed nickegg1018 closed 2 years ago

nickegg1018 commented 3 years ago

I wish I could give more pinpoint detail of this is the problem, but I've yet to really find a common denominator. I'm an HPC admin so I'm not the one doing the work. My user has run a bunch of E coli genomes through the pipeline and they seem to sporadically fail. I've attached a spreadsheet he made that details the run and what stages seemed to fail, as well as the log files from each of the runs. I know that probably won't be enough to determine what's happening but I figure it's a start, please let me know what else I can do to help troubleshoot. CMH-br-test-logs.zip

oschwengers commented 3 years ago

Hello @nickegg1018 , thanks for the detailed test reports. I wish I could come up with something more helpful but all I can extract from your examples is the following:

due to test 3, in principle the pipeline is installed correctly and is working as expected
test 1 & 2 reveil that in some cases the assembly step is failing.

This can occur due to plenty of reasons.

Have you used the same input data type for all three tests?
Which input data type have you used (Illumina, ONT, PB, Ill+ONT)?
Could you provide at least one of the assembly step log files so I can take a deeper look?

nickegg1018 commented 3 years ago

As I understand it, they were trying to test different input files, some being fastq and some being gbff format. I asked the user for the input type and he says Illumina paired end reads. I have easy access to the test 2 and 3 assembly logs and I've attached those. It'll take me a little bit to get the test 1 data so let me know if you'd still find that helpful. assembly-steps.zip

nickegg1018 commented 3 years ago

@oschwengers Is there anything else I can get for you to help debug this?

oschwengers commented 3 years ago

@nickegg1018 sry for the late reply. There are multiple quite random errors occuring within the execution of SPAdes at the assembly step. This is a hint that this is a rather technical issue and indeed I found a quite simple reason for the failed assemblies: it seems that there is not available HDD space available:

Assembly log for isolate SCB4 in test 3: line 1310: [Errno 28] No space left on device: '/var/scratch/tmp-1611953491940-883/warnings.log'

Could you please try to free some HDD space and retry executing ASA³P but also providing a distinct path for temporary data, hence it is not stored within the container itself via the -s <scratch_dir> parameter: $ #<ASAP_DIR>/asap-docker.sh -p <PROJECT_DIR> [-s <SCRATCH_DIR>] [-a ASAP_DIR] [-z] [-c] [-d]

Please, let me know if this solves the issue. Best regards