Closed jwiencek3 closed 7 years ago
Hi @jwiencek3, i'm working to reproduce this. I just did a quick sanity check to see if it's detecting tombstones at all and was able to:
in cqlsh:
create table and create row-level and cell-level tombstones:
cqlsh> create KEYSPACE simple WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
cqlsh> use simple;
cqlsh:simple> create table tbl (k int primary key, v int);
cqlsh:simple> create table tbl (k int primary key, v int);
cqlsh:simple> insert into tbl (k, v) values (0, 1);
cqlsh:simple> delete from tbl where k = 0 ;
cqlsh:simple> insert into tbl (k, v) values (1, 0);
cqlsh:simple> delete v from tbl where k = 1;
/.../node1/data0/simple/tbl-2d00e5516c0111e79e27ddcd9a2c9d1b/mc-1-big-Data.db
====================================================================================================
Partitions: 2
Rows: 1
Tombstones: 2
Cells: 1
Widest Partitions:
[1] 1
[0] 0
Largest Partitions:
[1] 37 37 B
[0] 0 0 B
Tombstone Leaders:
[1] 1
[0] 1
sstabledump output:
[1]@0 Row[info=[ts=1500413826297213] ]: | [v=<tombstone> ts=1500413838802882 ldt=1500413838]
[0]@32 deletedAt=1500413809480066, localDeletion=1500413809
Although it's entirely possible there's a case i'm missing. Will think about that.
Ah, i had an idea, maybe we aren't checking for Row level deletions. In my test case above i'm doing cell and partition level deletion, but nothing at row level.
That seems to work as well:
cqlsh> use simple;
cqlsh:simple> create table tbl2 (k0 int, k1 int, v int, primary key (k0, k1));
cqlsh:simple> insert into tbl2 (k0, k1, v) values (0, 0, 0);
cqlsh:simple> delete from tbl2 where k0=0 and k1=0;
/.../node1/data0/simple/tbl2-4fa2f1f16c0311e79e27ddcd9a2c9d1b/mc-1-big-Data.db
=====================================================================================================
Partitions: 1
Rows: 1
Tombstones: 1
Cells: 0
Widest Partitions:
[0] 1
Largest Partitions:
[0] 25 25 B
Tombstone Leaders:
[0] 1
[0]@0 deletedAt=1500414744019701, localDeletion=1500414744
[0]@0 Row[info=[ts=-9223372036854775808] del=deletedAt=1500414751075742, localDeletion=1500414751 ]: 0 |
I'll keep experimenting but any further info about what level the deletes are being done at would be helpful.
I believe I've reproduced the issue. It happens when you have a compound primary key and the partition key is also composed of several columns, i.e.:
cqlsh> use simple;
cqlsh:simple> create table tbl5 (k0 int, k1 int, k2 int, v int, primary key ((k0, k1), k2));
cqlsh:simple> insert into tbl5 (k0, k1, k2, v) values (0, 0, 0, 0);
cqlsh:simple> delete from tbl5 where k0=0 and k1=0 and k2=0;
/.../node1/data0/simple/tbl5-da5725316c0511e79e27ddcd9a2c9d1b/mc-1-big-Data.db
=====================================================================================================
Partitions: 1
Rows: 1
Tombstones: 0
Cells: 0
Widest Partitions:
[0:0] 1
Largest Partitions:
[0:0] 25 25 B
I'll see if I can fix this quickly.
Think i have a fix in #60. If that looks right let me know and i'll make a release, thanks!
I'll test it this morning and get back to you as soon as I can.
Regards
John
I tested the fix and it works as expected. Tombstone count matches total when I do a sstabledump -s
Awesome, thanks for verifying! I'll do a release tonight 👍
java -jar ~/sstable-tools-3.9.0-alpha9.jar cqlsh
cqlsh> describe sstables
/data/data/ionic_na/logs-a776ecc131b511e7bfae1bf589cdc0b2/mc-22945-big-Data.db ============================================================================== Partitions: 84792 Rows: 84792 Tombstones: 0 Cells: 1086408
Tombstones returns a value =0
When I run an sstabledump -d |grep 'deletedAt"
I get a very large number of rows returned.