search

data61 / gossamer

Gossamer bioinformatics suite
Other
19 stars 18 forks source link

xenome indexing hangs #33

Open ChristineMarie opened 2 years ago

ChristineMarie commented 2 years ago

I am trying to generate new xenome indices with the latest human and mouse assembly files. I am running on a 240 GB machine using 24 cores. However, the software seems to hang when writing out the edges. I let the software run for 3 days but it never completed processing. Any idea how to fix this issue?

tbrunetti commented 2 years ago

@ChristineMarie I am having weird issues as well. It is taking a really long time to index two genomes and it has been hanging for a full day. Were you ever able to get this to work?

ChristineMarie commented 2 years ago

My issue ended up being an out of space issue, however the xenome software will hang rather than giving that error (at least in my case). I had to mount an additional storage drive to my VM, and then also re-direct the temp directory to a tmp folder I created on that external volume. Otherwise the temporary files would fill up the disk space on my VM. Hope that helps

WillieYu commented 1 year ago

From my experience, giving it more memory and threads leads to a "caught unexpected exception: std::bad_alloc" error. I am running Xenome index for human host and mouse graft using an apptainer container I created on a server with 192 CPUs and 1TB of RAM and giving Xenome MAX_RAM=100GB and MAX_THREADS=48. The program did not respect the MAX_RAM=100GB setting during the "sorting the hashtable ..." step and consumed the entire 1TB of ram before crashing out due to "caught unexpected exception: std::bad_alloc" error. Setting the MAX_RAM to 900GB results in the same error. However, I set MAX_RAM=4 and MAX_THREADS=8 and the program ran just fine respecting the 4GB mem limit with ~12GB of temporary outputs in my temp folder during the "merging temporary graphs" stage. I guess the take-away is to use a lower resource setting when running the Xenome index.