xapian.DatabaseModifiedError exception is unhadled

[GoogleCodeExporter]

In a heavy update-during-search usage pattern when Xapian Database is being 
updated by process A (e.g. 'index' command), and another process B 
(Django+Djapian powered site) is being using the database for search, Xapian 
raises 'DatabaseModifiedError' exceptions periodically.

When 'DatabaseModifiedError' exception occurs, the Xapian Database instance 
must be reopened by using its 'reopen()' method.

Original issue reported on code.google.com by esizi...@gmail.com on 21 Oct 2011 at 12:45

[GoogleCodeExporter]

I've committed the initial solution in r382. I didn't manage to create a test 
case for the fix yet as it would require multiprocessing mess. Contribution of 
good test cases is highly appreciated.

Original comment by esizi...@gmail.com on 21 Oct 2011 at 12:48

[GoogleCodeExporter]

I have also seen this exception on 'document.get_value()' call, where 
'document' is an instance of xapian.Document. There are several lines in the 
code which could raise such an exception:
 * djapian/deciders.py:95:doc_value = document.get_value(self._values_map[field])
 * djapian/indexer.py:85:value = document.get_value(self.number)
 * djapian/resultset.py:223:model = doc.get_value(2)
 * djapian/resultset.py:225:pk = model._meta.pk.to_python(doc.get_value(1))

Other possible line where the exception may occur:
 * djapian/database.py:38:return self.open().get_doccount() 

I think we must wrap all of this with 'try: ... except: 
xapian.DatabaseModifiedError: ...' too.

Original comment by esizi...@gmail.com on 24 Oct 2011 at 11:18

[GoogleCodeExporter]

I agree.

Original comment by daevaorn on 24 Oct 2011 at 7:43

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

In r386 moved try:except block to a decorator, and guarded
Xapian.Database.get_doccount() call inside djapian.Database.document_count()

It's going to be non-trivial task to guard all document.get_value() call 
because there are no Database context here usually.

P.S. Initial commit was r383 actually, not r382.

Original comment by esizi...@gmail.com on 17 Nov 2011 at 10:16

[GoogleCodeExporter]

It looks like the problem has gone away with the current fix. At least I don't 
have problems anymore. Let's wait for a couple months more and I'll close the 
issue as fixed.

Original comment by esizi...@gmail.com on 23 Jan 2012 at 4:17

• Changed state: Accepted