search

Blosc / c-blosc2

A fast, compressed, persistent binary data store library for C.
https://www.blosc.org
Other
439 stars 82 forks source link

DOSHUFFLE bit in metainfo seems wrong (DeepState fuzzing) #94

Open agroce opened 4 years ago

agroce commented 4 years ago

DOSHUFFLE metainfo bit seems wrong:

RACE: Initialized test input buffer with data from `doshuffle.fail`
TRACE: Running: CBlosc2_RoundTrip from TestCBlosc2.cpp(34)
TRACE: TestCBlosc2.cpp(45): Performing 1 round trips.
TRACE: TestCBlosc2.cpp(48): *******************************   Starting run #0   *******************************
TRACE: TestCBlosc2.cpp(50): Type size = 1
TRACE: TestCBlosc2.cpp(52): Number of elements = 1
TRACE: TestCBlosc2.cpp(54): Buffer alignment = 32
TRACE: TestCBlosc2.cpp(56): Compression level = 0
TRACE: TestCBlosc2.cpp(58): Do shuffle = 1
TRACE: TestCBlosc2.cpp(61): Setting compressor to blosclz
TRACE: TestCBlosc2.cpp(65): Setting delta to 0
TRACE: TestCBlosc2.cpp(69): Buffer size = 1
TRACE: TestCBlosc2.cpp(89): # uncompressed bytes = 1
TRACE: TestCBlosc2.cpp(91): # compressed bytes = 33
TRACE: TestCBlosc2.cpp(92): block size = 1
TRACE: TestCBlosc2.cpp(97): type size = 1
TRACE: TestCBlosc2.cpp(99): DOSHUFFLE: 0
CRITICAL: TestCBlosc2.cpp(100): do shuffle = 0 but set to 1
ERROR: Failed: CBlosc2_RoundTrip
ERROR: Test case doshuffle.fail failed
agroce commented 4 years ago

Note this is REALLY rare:

CRITICAL: TestCBlosc2.cpp(100): do shuffle = 0 but set to 1
ERROR: Failed: CBlosc2_RoundTrip
INFO: Done fuzzing! Ran 195077 tests (3251 tests/second) with 459 failed/194618 passed/0 abandoned tests
agroce commented 4 years ago

The test above is fully reduced.

agroce commented 4 years ago

This does not go away once I restrict type_size to max 255

FrancescAlted commented 4 years ago

Yeah, here the issue is due to the fact that when typesize is 1, the shuffle is doing nothing, so I decided to silently switch off the filter internally. But again, this makes the API unconsistent as you have shown. Probably what we can do is to just check internally that if typesize is 1, just do not call the shuffle filter, but still keep the shuffle flag active. Thanks!

agroce commented 4 years ago

Great! Is there any functionality other than what's in this harness you'd really like to see fuzzed? I started from https://github.com/Blosc/c-blosc2/blob/master/tests/test_compress_roundtrip.c and called any obviously core API elements I could think of and check something about.