cdiener
commented
3 months ago Hi, I'm sorry you are experiencing this, it sure sounds frustrating. The multiprocessing module in Python can sometimes be a bit unstable. We sometimes see similar problems and they are often related to individuals processes generating a lot of log output that sometimes makes the thread pool unresponsive. The following might help (I would try those in that error):
- Make sure that build gets called in the main method, similar to the first example here.
- Switch out the multiprocessing method to "forkserver" or "spawn" with
import multiprocessing
...
if name == "main": multiprocessing.set_start_method("spawn")
...
3. Run some of the later samples with threads=1 to see error messages that have been missed before.
For RAM you should plan with 1-2GB per thread, so in your case I would start by allocating 56GB and then reduce that based on actual use later.
Hopefully one of those will help.
zoey-rw
Problem description
After creating a custom database, the "build" command sometimes stalls at 99% when creating a manifest from 100+ samples and 16 taxa. If I open a new Python session and run the same build command, it occasionally creates the manifest successfully. I'm also encountering stalling at 99% complete with the fix_medium command (same dataset).
I'm using 28 cores so I don't think it's a lack of RAM, but would appreciate any ideas for debugging!
Code Sample
Context
Other session info:
The beginning of the tax file looks like this, with 1410 rows total: