search

tjake / Solandra

Solandra = Solr + Cassandra
Apache License 2.0
882 stars 150 forks source link

Lucandra Index corruption #47

Closed heyvishy closed 13 years ago

heyvishy commented 13 years ago

Hi Jake, I am using Lucandra in my application. We are creating some rules in our application and I index them on the fly in Lucandra. As rules change, i need to update the corresponding index as well. But after a few times, the index gets corrupted. Scenario: When i update the lucandra document using updateDocument(updateTerm,doc,analyzer) method. It works for 1st , 2nd time, but it corrupts the indexed fields' value on 3rd time. I have verified that the value which i am inserting (on 3rd time ) is correct. I am using a web based UI tool (CassUI) ) for Cassandra to see/verify the DB values in Lucandra document. So , although i can see the value is lying in the lucandra document. But when i query for the same, i do not get any result. The indexSearcher is not able to search that document( With my query mentioning the value inserted on 3rd time). On debugging i verified that proper indexed value is getting inserted in the document. But somehow something else is getting corrupted, because the indexSearcher is not able to find that term in the document.

Is there a way to diagnose corrupt index?? Please help , i am stuck with this problem !!

tjake commented 13 years ago

Can you tell me what branch of lucandra you are using? What version of cassandra? and if you are calling reopen on your IndexReaders after each query?

heyvishy commented 13 years ago

Hi Jake, Thanks for getting back. I am using Master branch (which was updated on Nov 2 for issue #37) I am using Cassandra 0.6.4 version. Also i have a search function which takes query as input. In the function i am creating a fresh IndexReader object as below. IndexReader indexReader = new IndexReader("bookmarks", client);

So basically everytime the function is called, a new instance of indexReader object is created , which i think is same as indexReader re-open ?

Let me know if some more information is required. Thanks Vishal Shukla

tjake commented 13 years ago

Let me try to reproduce this, but it sounds like a bug in the 0.6 code

heyvishy commented 13 years ago

Hi Jake, Recently we migrated to Cassandra 0.7 version. So I installed Lucandra 0.7 version, and found that the index corruption bug stills exists in this version too. I replicated the same scenario ( as i had mentioned in my first mail) , and it consistently failed. Any ideas or suggestions would be highly appreciated.

tjake commented 13 years ago

Could you try the latest version as of this morning?

tjake commented 13 years ago

I think the latest commit to 0.7 branch fixes this problem by refetching if it encounters deleted data

heyvishy commented 13 years ago

Yes the fix works.I tested out the scenario where it used to consistently fail before , but it's working perfectly fine now with the latest code of 0.7 version. Thanks Jake.