search

rickhelmus / patRoon

Workflow solutions for mass-spectrometry based non-target analysis.
https://rickhelmus.github.io/patRoon/
GNU General Public License v3.0
65 stars 19 forks source link

Request: 'in memory'-option #84

Open LeonSaal opened 1 year ago

LeonSaal commented 1 year ago

Hi Rick,

I think it's great, that the package is so memory-efficient and lots of stuff is cached. However, I would appreciate, if there was an option to have spectra loaded 'in memory' for faster access times. I have lots of RAM at my disposal and I would like to put it to good use and save me some time ;) Thank you for your great work!

Kind reagrds, Leon

rickhelmus commented 1 year ago

Hi Leon,

That is a nice luxury ;-) Which steps are you specifically talking about? The peak lists are actually kept in memory after being generated, and during their creation all the spectra from a file are loaded at once into memory.

The EICs are a different story though, they are indeed loaded from cache whenever needed. I have some (still vague) ideas to change this in the future, but this is a bit tricky to do so ...

Thanks, Rick

LeonSaal commented 1 year ago

Hi Rick,

thanks for the fast reply and the insight on what is already stored in memory! As I cannot grasp for everything, what is stored where I am not sure, what could be done in memory and can be accelerated. Operations that take longer are every operation that begins with the message Verifying if your data is centroided... and I simply assumed, that is due to something being loaded from disk. E.g. loading MS peak lists e.g. when calling generateMSPeakLists takes a long time.

A little sidenote on the message Verifying if your data is centroided... : Even after the raw data had been accessed for feature finding, at later operations (e.g. during the generation of peak lists), the message pops up. The message and probably an underlying operation seems redundant, as it has already been verified at an earlier point. If something else happens in the background e.g. loading from cache, another message would be helpful.

Kind regards, Leon

rickhelmus commented 1 year ago

Hi Leon,

Funnily enough I actually pushed a small change regarding the centroiding verification. You can now disable this completely:

options(patRoon.checkCentroided = FALSE)

I did make these checks more efficient in the recent 2.2 release, but apparently they can still add significant overhead. I will see if I can come up with a better default, but this should suffice as a workaround.