MEGAHIT didn't run well - OSError: [Errno 2] No such file or directory: /tmp/k29/29.sdbg.24

[mdebiasse]

Hello! I am running MitoFinder v1.4.1 installed with conda on a linux server and my job is failing during MEGAHIT. My command:

module load anaconda3/2021.05
source activate mitofinder 
cd /home/mdebiasse/borgstore/dlab/01-CCGP/03-SCARIOSUS/08-MITOFINDER
/home/mdebiasse/.conda/envs/mitofinder/bin/mitofinder -j semi_cari -1 /home/mdebiasse/borgstore/dlab/01-CCGP/01-DATA/18-SCARI_OMNIC/ftps.fulgentgenetics.com/SE8481/SA190613_14_trim_1.fastq -2 /home/mdebiasse/borgstore/dlab/01-CCGP/01-DATA/18-SCARI_OMNIC/ftps.fulgentgenetics.com/SE8481/SA190613_14_trim_2.fastq -r NC_039849.gb -o 5 -p 28 -m 240

The output file says MEGAHIT didnt run well. The megahit.log states:

2023-05-10 13:33:21 - MEGAHIT v1.2.9
2023-05-10 13:33:21 - Using megahit_core with POPCNT and BMI2 support
2023-05-10 13:33:22 - Convert reads to binary library
2023-05-10 13:44:53 - INFO  sequence/io/sequence_lib.cpp  :   75 - Lib 0 (/home/mdebiasse/borgstore/dlab/01-CCGP/01-DATA/18-SCARI_OMNIC/ftps.fulgentgenetics.com/SE8481/SA190613_14_trim_1.fastq,/home/mdebiasse/borgstore/dlab/01-CCGP/01-DATA/18-SCARI_OMNIC/ftps.fulgentgenetics.com/SE8481/SA190613_14_trim_2.fastq): pe, 740058186 reads, 151 max length
2023-05-10 13:44:53 - INFO  utils/utils.h                 :  152 - Real: 691.9458   user: 510.5266  sys: 61.5730    maxrss: 250200
2023-05-10 13:44:53 - k-max reset to: 141
2023-05-10 13:44:53 - Start assembly. Number of CPU threads 28
2023-05-10 13:44:53 - k list: 21,29,39,59,79,99,119,141
2023-05-10 13:44:53 - Memory used: 240000000000
2023-05-10 13:44:53 - Extract solid (k+1)-mers for k = 21
2023-05-10 14:03:56 - Build graph for k = 21
2023-05-10 14:07:56 - Assemble contigs from SdBG for k = 21
2023-05-10 15:25:26 - Local assembly for k = 21
2023-05-10 15:42:31 - Extract iterative edges from k = 21 to 29
2023-05-10 16:06:34 - Build graph for k = 29
2023-05-10 16:09:57 - Assemble contigs from SdBG for k = 29
Traceback (most recent call last):
  File "/home/mdebiasse/.conda/envs/mitofinder/bin/megahit", line 1038, in <module>
    main()
  File "/home/mdebiasse/.conda/envs/mitofinder/bin/megahit", line 1015, in main
    assemble(next_k)
  File "/home/mdebiasse/.conda/envs/mitofinder/bin/megahit", line 264, in checked_or_call
    func(*args, **kwargs)
  File "/home/mdebiasse/.conda/envs/mitofinder/bin/megahit", line 904, in assemble
    remove_temp_after_assemble(cur_k)
  File "/home/mdebiasse/.conda/envs/mitofinder/bin/megahit", line 656, in remove_temp_after_assemble
    remove_if_exists(graph_prefix(kmer_k) + '.sdbg.' + str(i))
  File "/home/mdebiasse/.conda/envs/mitofinder/bin/megahit", line 127, in remove_if_exists
    os.remove(file_name)
OSError: [Errno 2] No such file or directory: '/mnt/borgstore/mdawson/dlab/01-CCGP/03-SCARIOSUS/08-MITOFINDER/semi_cari/semi_cari_megahit/tmp/k29/29.sdbg.24'

When I cd into /tmp/k29/ I see these files:

29.edges.info  29.sdbg.25  29.sdbg.26  29.sdbg.27  29.sdbg_info

I can provide the record of the conda install if that is useful. Any advice to correct this error is appreciated! Thank you!

[RemiAllio]

Hi! Thank you for reaching out.

Unfortunately, I have never used MitoFinder via Conda and this error comes from MEGAHIT itself, so I don't know how to help. I know that there is a conda environment available but I didn't create it..

However, do you know if singularity is installed on your server or can be easily installed? This would be the easiest way to use MitoFinder without any other dependency issues.

Let me know and we'll see what we can do! Sorry for the inconvenience, Best, Rémi

[mdebiasse]

Hello Rémi, thank you for the reply! I ran MitoFinder with singularity and the job finished successfully. Interestingly, the first time I launched the job with singularity, the command to bind my working directory was incorrect so the output went to my /home dir, and the fun failed with a similar MEGAHIT error. Once I corrected the bind command, MitoFinder ran successfully. I was hitting a storage quota when the output was going to my /home dir and I suspect that terminated the run prematurely although the MEGHIT hit error message did not indicate this specifically. Just thought I would mention it in case other folks run into this issue. Thanks for providing a great program with MitoFinder!

[RemiAllio]

Hi!

Glad it worked with the singularity container! Thank you for sharing your experience and for you positive feedback! :slightly_smiling_face:

Best, Rémi