search

rietho / IPO

A Tool for automated Optimization of XCMS Parameters
http://bioconductor.org/packages/IPO/
Other
34 stars 20 forks source link

IPO parallelizing not working, large scale dataset hundreds to thousands of samples #64

Open victorfrnak opened 4 years ago

victorfrnak commented 4 years ago

I am running IPO on a set of 200 + samples (and will have a set of 1800 + later). So, the parallelizing options are very important. However, the IPO run seems to be taking a long time (over a week) so I believe that the parallelization is not working. When I stop the R command, I get this message. Even though I am using BPPARAM at 10.

IPO (nSlaves-argument) and xcms (BPPARAM-argument) parallelisation cannot be used together!

Am I correct in believing that the parallelization is not working? Is it feasible to run IPO on hundreds of samples?

sneumann commented 4 years ago

Hi, suggestion from my side: there is no need to run the optimisation on all samples. Instead, you could pick 20-100 QC samples, and use the resulting parameters to guide your processing of the 1800 samples. Yours, Steffen

victorfrnak commented 4 years ago

Hi Steffen. thanks for the reply. I have thought about that - however, I am particularly interested in optimizing with the correct determination of the minfrac parameter ... (fraction of samples a metabolite must be in) and I would like to calculate this as a function of the entire dataset. Is it possible to run IPO at this scale? Sorry for not getting back to this until now, I was preparing a talk. Thanks

victorfrnak commented 4 years ago

Given this error: "IPO (nSlaves-argument) and xcms (BPPARAM-argument) parallelisation cannot be used together! Setting nSlaves to 1. " Is there an inherent conflict between xcms and IPO in the way that IPO is running xcms functions?