Open ThomasPouplin opened 4 years ago
Sorry to ask this question but I'm a bit lost in parallel computing ...
Could somebody explain the ratio between workers and slaves for running IPO optimally ?
I have access to clusters (>200 CPUs) and would like to optimise the run time on my lipidomics datasets (POS and NEG).
Let's say I use 6 QCs samples to perform the optimization.
I understood the following formula: Cores = IPO-slaves * XCMS slaves
Where I'm really lost, is where are these workers/slaves described between the Snowparam (n workers) and the optimizeXcmsSet (n slaves).
How can I, for example, set one slave per sample (so 6 slaves), and split workers for each slaves (say 4 workers per slave) ?
At the moment, I just set 8 workers and 8 salves - IPO runs without any errors but maybe not optimally.
This is a part of the code I'm using (on 8 CPUs):
snowparam <- SnowParam(workers = 8, type = "SOCK") peakpickingParameters = getDefaultXcmsSetStartingParams('centWave') # get defaults
resultPeakpicking = optimizeXcmsSet( params = peakpickingParameters, isotopeIdentification = "IPO", nSlaves = 8, subdir = 'CentWave' )
Thanks for your feedback !
Sorry to ask this question but I'm a bit lost in parallel computing ...
Could somebody explain the ratio between workers and slaves for running IPO optimally ?
I have access to clusters (>200 CPUs) and would like to optimise the run time on my lipidomics datasets (POS and NEG).
Let's say I use 6 QCs samples to perform the optimization.
I understood the following formula: Cores = IPO-slaves * XCMS slaves
Where I'm really lost, is where are these workers/slaves described between the Snowparam (n workers) and the optimizeXcmsSet (n slaves).
How can I, for example, set one slave per sample (so 6 slaves), and split workers for each slaves (say 4 workers per slave) ?
At the moment, I just set 8 workers and 8 salves - IPO runs without any errors but maybe not optimally.
This is a part of the code I'm using (on 8 CPUs):
snowparam <- SnowParam(workers = 8, type = "SOCK") peakpickingParameters = getDefaultXcmsSetStartingParams('centWave') # get defaults
resultPeakpicking = optimizeXcmsSet( params = peakpickingParameters, isotopeIdentification = "IPO", nSlaves = 8, subdir = 'CentWave' )
Thanks for your feedback !