search

domo141 / nottoomuch

Misc material I use with notmuch mail indexer
32 stars 3 forks source link

Handling simultaneous access while database modification #2

Closed suvayu closed 4 years ago

suvayu commented 10 years ago

Not sure if this can really be done any better. I was doing a rebuild of the database, when a cron job ran offlineimap and notmuch new. This made the database rebuild fail prematurely like this:

$ ./nottoomuch-addresses.sh --update --rebuild
Creating '/home/user/.config/nottoomuch/addresses'. This may take some time...
terminate called after throwing an instance of 'Xapian::DatabaseModifiedError'
Added 6786 active addresses in 139 seconds.
Total number of active addresses: 6786.
domo141 commented 10 years ago

well, it can -- first get file list to a temporary file and then start processing it...

domo141 commented 10 years ago

the version in 'dogfood' branch now uses temporary file -- it shortens the run time of notmuch search considerably!

https://raw.githubusercontent.com/domo141/nottoomuch/dogfood/nottoomuch-addresses.sh

suvayu commented 10 years ago

Thank you. I'll check it out.

domo141 commented 4 years ago

Now, version 2.5 of nottoomuch-addresses.sh (well, 2.4 already) uses that temporary file so database is locked for shorter period of time.

Ultimately, the locking of db could be made shorter by dumping all output to a file, and after notmuch search has exited, start filtering ignores out.

Since --update / --rebuild is supposed to be done manually (and not too often), and, if there is chance for (unrecoverable) database failure anyway, to improve things further, change to notmuch (i.e. retry in case of such a known potential failurre) should be done instead (if not already done!)

closing as, IMO, solution good enough.