deanhiller / playorm

ORM for nosql with SCALABLE SQL
https://github.com/deanhiller/playorm/wiki
Other
76 stars 18 forks source link

removing entity with field anotated @NoSqlId @NoSqlIndexed will not remove index entry #123

Closed hsn10 closed 11 years ago

hsn10 commented 11 years ago

have testcase for this

hsn10 commented 11 years ago

https://github.com/hsn10/ormtest

branch indexnotdeleted

easility commented 11 years ago

is it same as #116? if not, what error you are getting?

hsn10 commented 11 years ago

i think these are 2 different problems. Primary problem is this and it makes other problem to appear.

No error, Index entries just sits there. run example with: sbt run

easility commented 11 years ago

Somehow, I got the same problem as #116. Nevertheless, I have added a check..and the error is not coming not now..can you check on latest code of PlayOrm.

hsn10 commented 11 years ago

This is still not fixed, entry in IntegerIndice is not deleted.

easility commented 11 years ago

ohk..you want entry from IntegerIndice to be gone..(though they are not visible on our viewindex tool)..I looked into it..In my case, the deletion is not deleting the rowkeys though columns in the rows are deleted... i.e., when I do "list NoteRead" on Cassandra's CLI tool..the output it gave is

 RowKey: 2970768600550878517
 -------------------
RowKey: 4513768224221392554
  -------------------
 .
 . 
  .
  RowKey: 8208888952628063913
  -------------------
  RowKey: 3297174875194081006

 100 Rows Returned.
 Elapsed time: 571 msec(s).
 [default@ormtest2]

So may be that is the reason our tool is not deleting them..seems that we need to run compaction to delete them completely. see this link http://stackoverflow.com/questions/9265795/how-to-force-final-removal-of-deleted-rows-in-cassandra?rq=1

I am looking more into it..but hope that it is not critical now..

hsn10 commented 11 years ago

Not deleting rowkeys is standard cassandra behaviour, there is nothing wrong with it. If you delete row it creates tombstone. tombstones are listed in list command. Tombstone gets deleted during compaction after 10 days by default.

I do not complain about tombstones in NoteRead. Problematic are entries remaining in IntegerIndeces - they point to tombstoned rows - they are useless.

Entry from IntegerIndice will get deleted if you annotate with @NoSqlIndexed anything other then primary row key.

hsn10 commented 11 years ago

I think that even after 10 days which makes tombstoned rows to disappear, old entries in IntegerIndeces will be still there wasting disk space.

deanhiller commented 11 years ago

Vikas, @hsn10 is correct, it is not that cassandra does not delete them write away. Our other indexes delete just fine...it is only the primary key one (well, I have not verified myself but I understand what he is saying as I originally tested that with fields but never tested it with pk).

All, do we have a test case that shows failure on this one checked in? There should be a test case that is getting a RowNotfoundException or something as the index is there but the row is gone(actually row is tombstoned not completely gone but appears gone from client and index is still there when it should have been tombstoned).

hsn10 commented 11 years ago

MetaClassSingle.hasIndexedField() whis is used for index deletion appears to be coded right.

hsn10 commented 11 years ago

i am trying to write unit test for this.

I need to call em.getSession().scanIndex but first parameter is ScanInfo and it needs DboTableMeta data for its constructor. where can i find them? They are private (MetaInfo) in BasicEntityManagerImpl and no method for getting them.

What API returns DboTableMeta? i have to start from EntityManager.

easility commented 11 years ago

One way is to get like this: DboTableMeta table = mgr.find(DboTableMeta.class, "PartAccount"); You can view the related code in testIndexedButNotInNoSqlDatabaseList() of TestIndexes

deanhiller commented 11 years ago

yes, loading our meta data is just like loading your own entites ;) and no different. the meta data was made into entites for storage and lookup for ease of use.