search

baskard / thrudb

Automatically exported from code.google.com/p/thrudb
0 stars 0 forks source link

Race condition / crash in Thrudex when adding items #5

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?
It's a bit hard to reproduce this.
1. Add something to an index.
2. Query the item within 1-2 seconds
3. Sometimes, boom!

When this happens Thrudex either crashes outright or gets wedged. Often the
new version of the data never gets merged in.

I am using the 20 Dec SVN on the EC2 AMI. I have not been able to get any
later versions of the SVN to compile.

64337 [2553588624] DEBUG CLuceneBackend  - put: 3RPPyo1ZFAt6
64337 [2553588624] DEBUG CLuceneBackend  - userid:vyKatKMeMRYX
64337 [2553588624] DEBUG CLuceneBackend  - Text
64337 [2553588624] DEBUG CLuceneBackend  - groupid:0
64337 [2553588624] DEBUG CLuceneBackend  - Text
65430 [2552535952] DEBUG CLuceneIndex  - Searching in: (table)
TNonblockingServer uncaught exception.
66399 [2552535952] DEBUG CLuceneIndex  - Created new searcher
66399 [2552535952] DEBUG CLuceneIndex  - userid:vyKatKMeMRYX
66399 [2558851984] DEBUG CLuceneIndex  - Created Handles
66400 [2558851984] DEBUG CLuceneIndex  - Deleted3RPPyo1ZFAt6
66400 [2558851984] DEBUG CLuceneIndex  - Deleted old ids
68293 [2558851984] DEBUG CLuceneIndex  - Merged
68293 [2558851984] DEBUG CLuceneIndex  - Query

Original issue reported on code.google.com by aris...@gmail.com on 25 Jan 2009 at 2:50

GoogleCodeExporter commented 8 years ago 
Turns out this is an issue with CLucene's impementation of non-compound 
indexes.  where the segment file 
lengths aren't maintained properly for each file and the index reader reads 
beyond the end of the file.

The fix here is to switch to compound indexes. which are actually better since 
they open less filehandles though 
there is a perf hit.

Original comment by jake%3.r...@gtempaccount.com on 31 Jan 2009 at 6:28