Closed DerLinkman closed 2 years ago
The memory shall recover after indexing or searching, so there seems to be a leaking somewhere
Can you tell which version of the plugin are you using ? Can you test with latest git ?
Hi yes sure.
We build the xapian plugin by ourselfs from the latest git. I rebuild it just recently, so it is commit: https://github.com/grosjo/fts-xapian/commit/c6e4ab0c3925d8fecb4d6205dccdfcfb4981461f
Hi, any update on this ?
Hi,
i´ll try it with the latest Git Commit https://github.com/grosjo/fts-xapian/commit/1a7361e1d1169f10546b6854cb335ed73d529bdd
I´ll keep you up to date!
Sorry, i´ll test this tomorrow :(
Hi i think it got even worse with the latest patch.
Now the plugin is always in low memory mode even if i allocated 2048 MB the configuration of the plugin is untouched.
Please let me know if you need something else from me.
Build: https://github.com/grosjo/fts-xapian/commit/13f08da2e1ed13412e600348ce72982f2163572f
Now the plugin is always in low memory mode even if i allocated 2048 MB the configuration of the plugin is untouched.
That was a false positive it is not running in lowmemory mode everytime, sorry for the confusion.
But the issue still seems to appear. If i reindexed the mails with doveadm fts optimize -A i get that in the logs: Info: FTS Xapian: Starting with partial=3 full=20 verbose=1 lowmemory=250 MB vs freemem=262 MB
.
I saw something that might have something to do with it: In htop the RAM of the dovecot/index-workers is only cleaned up after a few minutes, once this happens the RAM count of the fts-xapian log increases again.
Is this also the reason why the reindexing process goes into LowMemory mode after some time?
It will probably cache the files in RAM before writing them to disk or not? Then the RAM should be empty again after a write process to the disk theoretically (if it is so at all, unfortunately I'm not that deep in the matter :( ).
Hey,
can you explain me how Xapian handles the memory usage? I´m a bit confused tbh.
I´ve assigned 2048M for the index workers.
But Xapian says (when dovecot is freshly started) about a Gigabyte more than i´ve set (lowmem vs freemem)
After i start a reindex and some searches the freemem drops and won´t recover.
(Currently 450MB instead of 2048) what is the reason for that?
Is there a way to clean the Xapian "Cache" which flush the used Xapian Ram?
Thx for your help and your awesome plugin!
Kind regards Niklas