VladRodionov / bigbase

BigBase - read optimized, fully HBase-compatible, NoSQL Data Store
GNU Affero General Public License v3.0
9 stars 1 forks source link

Block cache: add block checksum support for external storage #11

Open VladRodionov opened 10 years ago

VladRodionov commented 10 years ago

We need additional check-sum to verify data integrity. When Linux has hardware failure, some garbage can be written to data file, that is why we need check-sums to verify data integrity on reading. When check-sum fails, file needs to be truncated to the position of an end of last valid block.

For check-sum generation, we have com.koda.util.Utils class (hash functions). We will probably need some additional code.