rickhelmus / patRoon

Workflow solutions for mass-spectrometry based non-target analysis.
https://rickhelmus.github.io/patRoon/
GNU General Public License v3.0
65 stars 19 forks source link

Request: Reduce cache size #98

Open LeonSaal opened 10 months ago

LeonSaal commented 10 months ago

Hi Rick,

do you think it is possible to reduce the size of the cache file during filling of the cache? It blows up quite quickly and clutters my harddrive. My current strategy is to delete the cache of previous evaluations and recache it if needed.

Kind regards,

Leon

rickhelmus commented 10 months ago

Hi Leon,

Good question! When I implemented the caching I had in mind to put limits, but in the end this didn't turn out to be very straightforward to do automatically. But I will keep it in mind and try to investigate what is possible.

For now, I guess you could always manually edit the SQLITE file and remove older rows.

Thanks, Rick

LeonSaal commented 6 months ago

Hi Rick,

a follow-up question: What is the cache-table _filterFGroupsminReplicates used for? For me it takes up ~ 60 % of the total size of the cache and I wonder why it it so much bigger, than the other _filterFGroups__-tables:

name tablesize
filterFGroups_minReplicates 4574308
specData 1517436
EICData 1480564
filterFGroups_replicateAbundance 24364
MSPeakListsMzR 17776
calculatePeakQualities 16732
featureGroupsXCMS3 16172
filterFGroups_blank 16004
filterMSPeakLists 15244
filterFGroups_replicate_group 12568
featureGroupsOpenMS 8688
featuresXCMS3 8428
mzREIC 6196
formulasGenForm 2320
MSPeakListsAvg 2180
componentsCAMERA 1084
formulasFGroupConsensus 936
sqlite_autoindex_mzREIC_1 128
sqlite_autoindex_MSPeakListsMzR_1 120
sqlite_autoindex_filterFGroups_minReplicates_1 60
sqlite_autoindex_formulasGenForm_1 32
sqlite_schema 4
sqlite_autoindex_specData_1 4
sqlite_autoindex_formulasFGroupConsensus_1 4
sqlite_autoindex_filterMSPeakLists_1 4
sqlite_autoindex_filterFGroups_replicate_group_1 4
sqlite_autoindex_filterFGroups_replicateAbundance_1 4
sqlite_autoindex_filterFGroups_blank_1 4
sqlite_autoindex_featuresXCMS3_1 4
sqlite_autoindex_featureGroupsXCMS3_1 4
sqlite_autoindex_featureGroupsOpenMS_1 4
sqlite_autoindex_dataCentroided_1 4
sqlite_autoindex_componentsCAMERA_1 4
sqlite_autoindex_calculatePeakQualities_1 4
sqlite_autoindex_MSPeakListsAvg_1 4
sqlite_autoindex_EICData_1 4
dataCentroided 4

Kind regards,

Leon