byzhang / leveldb

Automatically exported from code.google.com/p/leveldb
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

BloomFilterPolicy not interoperable between ARM and x86 #237

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
We've been struggling with an elusive issue for a while in which LevelDB 
database files were not compatible between platforms. Copied databases were 
failing in subtle ways undetected by any checks.

After a slough of debugging this was narrowed down to use of BloomFilterPolicy. 
The bloom filter uses the Hash function that can return different output based 
on char signedness. To be specific: if the number of characters is not a 
multiple of four, and at least one of the trailing characters is >0x80.

Attached is a patch that:
- Add casts to Hash() to treat chars as unsigned (I think this was intended)
- Adds tests for Hash
- Change name of algoritm in BloomFilterPolicy::Name to make sure that   
filters are recomputed; this provides forward and backward compatibility

Luckily, the use of Hash() in BloomFilterPolicy seems to be the only place 
where the result of that function affects what is written to disk.

Original issue reported on code.google.com by laa...@gmail.com on 7 Jun 2014 at 11:51

Attachments: