Fixed checksumming empty data

reverbrain / eblob

Eblob is an append-only low-level IO library, which saves data in blob files. Created as low-level backend for elliptics

GNU Lesser General Public License v3.0

104 stars 29 forks source link

Fixed checksumming empty data #163

Closed shaitan closed 8 years ago

shaitan commented 8 years ago

In eblob_chunked_mmhash size == 0 can be caused by only one of:

a client tries to verify 0 bytes from the data;
a client updates 0 bytes of the data and now eblob is calculating checksums of updated part;

in both cases checksums shouldn't be calculated because no data is touched or updated.

bioothod commented 8 years ago

Does it mean that we have checksums for zero data already in the storage? How eblob will read those keys now?

shaitan commented 8 years ago

Does it mean that we have checksums for zero data already in the storage?

No, it doesn't because zero data has no checksum.

How eblob will read those keys now?

It will read only header and fill result with zero data size.

bioothod commented 8 years ago

So basically this is an optimization for checksum verification - do not try to verify it if data size is zero? What does it read right now?

Looks like it converts last_chunk into very large number (-1 as 64bit unsigned long long), and if allocation succeeds, it reads data from another very large offset, which likely fails? Is that correct?

shaitan commented 8 years ago

Basically, it is fix for both verification and calculation - do not try calculate checksum if data size is zero. What read do you mean?

Yes, if both offset and size is zero, it set last_chunk to -1ULL earlier which led to huge memory allocation and checksums calculation for incorrect data.

bioothod commented 8 years ago

Ok, I see. Patch applied, thank you