Closed jfx319 closed 4 years ago
Hi,
You can use:
-t
option on all analysis steps that supports parallelization to override the auto-detected concurrency level-Xmx
option to specify amount of memory available to the processe.g.
mixcr align -t 4 -Xmx4g ...
MiXCR uses some empirical formulas to determine the amount of available memory (see script) and [availableProcessors()](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Runtime.html#availableProcessors()) method to determine number of CPUs.
Thanks!
Is there a way to pass such flags to the main wrapper? Or does each step need to be deconstructed?
mixcr -t 4 -Xmx16g analyze shotgun
... etc
If diving into specific steps is required, where can I find a vignette tutorial workflow that might help with understanding how to use each step and how to interpret the results of each step in the context of a real dataset?
mixcr analyze
will correctly pass both options to all the steps.
Minimal reproducible example:
mkdir test
cd test
wget https://github.com/milaboratory/milib/raw/f9219a0b31e37ec45c3cdb055bd1e8463499a3ff/src/test/resources/sequences/sample_r1.fastq.gz
wget https://github.com/milaboratory/milib/raw/f9219a0b31e37ec45c3cdb055bd1e8463499a3ff/src/test/resources/sequences/sample_r2.fastq.gz
#wasn't sure about species or rna vs dna (feel free to change)
mixcr -t 4 -Xmx4g analyze shotgun \
--species HomoSapiens \
--starting-material dna \
--receptor-type xcr \
sample_r1.fastq.gz \
sample_r2.fastq.gz \
test
ERROR:
Unknown options: -t, 4
looks like the -Xmx4g
flag works, but not the -t
flag; am I invoking it correctly?
Version info:
mixcr -v
MiXCR v3.0.12 (built Wed Nov 20 08:38:34 EST 2019; rev=cad6564; branch=release/v3.0.12; host=Dmitrys-MacBook-Pro.local)
RepSeq.IO v1.3.3 (rev=fd1bd59)
MiLib v1.12 (rev=12b1449)
Built-in V/D/J/C library: repseqio.v1.6
I'm really sorry, I misinformed you. I remembered that we implemented it, but in fact we didn't.
You should specify -t
option for each of the steps, like this:
mixcr -Xmx4g analyze --align "-t 4" ...
-Xmx4g
should be specified only once.
Just created an issue for this #563.
No problem, thanks for your help on this!
I will use the option you suggested in the mean time. Is there another step besides "align" that you think is worth specifying more threads to? On an initial test, I recall the align step seemed to take the vast majority of the time, but am not sure if another step (e.g. contig assembly) would benefit too. Looking through readthedocs, I only see --assemble
having the option. Let me know if I'm missing any.
I am guessing it would be:
numThreads=4
mixcr -Xmx4g analyze shotgun \
--align "-t ${numThreads}" \
--assemble "-t ${numThreads}" \
--contig-assembly \
--species HomoSapiens \
--starting-material dna \
--receptor-type xcr \
sample_r1.fastq.gz \
sample_r2.fastq.gz \
test
I tested this and it seems to work fine; more importantly, it now substantially decreases the predicted align ETA to ~34 hours for a WGS with 64 threads.
You should also add -t
option for contig-assembly (--contig-assembly "-t ${numThreads}"
).
We've fixed the problem in #563 and will include it in MiXCR 3.0.13 which will be released on the next week. We'll let you know. Thanks for reporting!
Thanks!
You should also add
-t
option for contig-assembly (--contig-assembly "-t ${numThreads}"
).
I tested this and it complains that there is no -t option (removing this from --contig-assembly fixes the complaint).
Fixed in 3.0.13, release on road
@jfx319 you wrote:
"When running mixcr on a... cluster (e.g. with slurm scheduler), it is single-threaded and ignores the extra cores reserved."
Can you please suggest how did you check that?
When running mixcr on a local machine (e.g. laptop), the program automatically uses all cores.
But when running on a cluster (e.g. with slurm scheduler), it is single-threaded and ignores the extra cores reserved.
Is there a more explicit way to manually specify parallelization & memory, rather than relying on presumably an auto-detect mechanism (that seems to fail in this HPC environment)? It's also difficult to use auto-detect on a shared node in HPC, since potentially the usual commands to check max mem/cores reveals more resources (the whole node) than actually reserved for this particular job (a fraction of the node).
Any advice would be appreciated!