search

jeffjirsa / twcs

Time Window CompactionStrategy for Apache Cassandra
52 stars 24 forks source link

Has twcs resolved the out-of-order problem #3

Closed FS1360472174 closed 8 years ago

FS1360472174 commented 8 years ago

actually when i use the DTCS,have not found a method to solve the out-of-order data problem. but read repair and hintedhandoff always inject the out-of-order data.we need read repair and hintedhandoff to keep the data consistency.

so i want to know has TWCS resolved the out-of-order problem?

jeffjirsa commented 8 years ago

TWCS handles out of order data differently than DTCS, but neither are perfect. In TWCS, we bucket sstables by max timestamp, not min, so read repair / hint data puts old data into a new file, which avoids write amplification, and has less blocking effect when sstables expire.

It's not perfect, the real fix is to split old data out on flush or on compaction, and there's an upstream ticket for that, but it's not ready yet. Will be done on the C* side, not in a compaction strategy itself.

jeffjirsa commented 8 years ago

Closing: TWCS handles this "better" than DTCS (in my opinion, unless someone can show me it's wrong), and the proper fix is upstream via https://issues.apache.org/jira/browse/CASSANDRA-10496