search

bokulich-lab / q2-assembly

QIIME 2 plugin for (meta)genome assembly.
BSD 3-Clause "New" or "Revised" License
4 stars 12 forks source link

`evaluate-contigs` fails with `OSError: [Errno 28] No space left on device` #39

Closed gregcaporaso closed 7 months ago

gregcaporaso commented 1 year ago

On two separate machines (both HPC clusters), I've had evaluate-contigs fail due to available disk space. I can see from the log that it downloads a lot of files. @misialq, have you run into this? Any ideas on how to address this? I haven't actually got this command to complete yet, after testing on two different studies (and two different systems, as I mentioned). Let me know if you'd like the error log - I can send that by email.

misialq commented 1 year ago

Hey @gregcaporaso, I suspect this is related to https://github.com/bokulich-lab/q2-assembly/issues/35 - QUAST is downloading some reference data in the background (which I realised only recently) so we need to make it possible to just provide those as input to the action itself (I haven't checked in detail whether that's possible or not but I certainly hope it is). Please do send the log file over, if you have it - I can have a look just to confirm whether it's this problem or something else entirely. Thanks!

gregcaporaso commented 1 year ago

I thought it might be related to #35 too - just sent you the error log by email.

colinvwood commented 1 year ago

Any updates? We ran into the same error today when using a tiny (4 sample) dataset.

gregcaporaso commented 1 year ago

QUAST is downloading some reference data in the background (which I realised only recently) so we need to make it possible to just provide those as input to the action itself (I haven't checked in detail whether that's possible or not but I certainly hope it is).

It is possible - we can provide those in fasta format.

colinvwood commented 1 year ago

It doesn't look like the space issue is due to downloaded references, but rather inefficiency in storing alignment files. See this and this.

misialq commented 1 year ago

Do you know whether you're running out space wherever the tmp directory is located or somewhere else? I think these are two separate issues: those alignment files should be stored in tmp (as far as I can tell) so if you're good on space there, the issue is somewhere else...

colinvwood commented 1 year ago

I know that this is redundant to what I posted on the forum, but responding for reference's sake. I can confirm that the alignment files are being stored in the tmp directory, and can confirm that that directory did run out of space on our cluster after ~500GB had been written to it.

misialq commented 7 months ago

I have not seen this happen with any of our runs so I'm going to close it for now (since we don't really have any way to address it on our end anyway)