Many issues uncovered by fuzzing

[nemequ]

I reported many issues which cause zling to crash on decompression found by AFL (via e-mail on 2015-02-11). They are still unfixed, and I think enough time has elapsed that it is better to disclose them publicly.

They have been uploaded to https://github.com/nemequ/compfuzz/tree/master/libraries/zling/codecs/zling/decompress/crashes

Some of them may require AddressSanitizer (or possibly a similar tool, such as valgrind) in order to trigger the crash on a specific machine. Some are probably exploitable, all are potential vectors for a DoS attack.

[richox]

can you test the latest version? i fixed some bugs but i'm not sure whether it can pass the fuzzy test now

[nemequ]

It's still vulnerable.

It's quite trivial to test; clone a copy of the compfuzz repository, then do something like

for source in path/to/compfuzz/libraries/zling/codecs/zling/decompress/crashes; do
  path/to/libzling/build/zling_demo d "$source" /dev/null;
done

It's fine if zling fails (most of the archives are probably invalid), but not if it crahes. You should also test against a copy of zling compiled with AddressSanitizer (just add -fsanitize=address to your compiler flags for clang or gcc), and/or run them in valgrind.

[richox]

i made some fix and now it won't crash on compfuzz, if you run with valgrind, there may be some access on non-initialized memory, it should be acceptable

[lhmouse]

@richox

there may be some access on non-initialized memory, it should be acceptable

That is laughable. Period.

[FrankHB]

@richox On what?

[richox]

@lhmouse too young too simple, sometimes naive

[nemequ]

i made some fix and now it won't crash on compfuzz, if you run with valgrind, there may be some access on non-initialized memory, it should be acceptable

Okay, if I turn off ubsan it works. It would be nice if there were a way to initialize that memory, even if it's off by default (perhaps behind an #ifdef?). UBSan and valgrind can pick up some real bugs, it would be a shame if false (kind of…) positives prevented them from being used. zling is already extremely slow for small buffers, I doubt memset()ting some memory would make much of a difference…

When setting up another run of AFL I actually managed to find a crash in the compressor. Try compressing testimage.ppm at level 3 or 4. If ASan is enabled you should see something like

=================================================================
==27776==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f81d6cfcbbc at pc 0x7f81db4911d5 bp 0x7ffce2a56450 sp 0x7ffce2a56440
READ of size 4 at 0x7f81d6cfcbbc thread T0
    #0 0x7f81db4911d4 in baidu::zling::lz::ZlingRolzEncoder::MatchLazy(unsigned char*, int, int, int) /home/nemequ/local/src/libzling/src/libzling_lz.cpp:266
    #1 0x7f81db4911d4 in baidu::zling::lz::ZlingRolzEncoder::MatchAndUpdate(unsigned char*, int, int*, int*, int) /home/nemequ/local/src/libzling/src/libzling_lz.cpp:243
    #2 0x7f81db4911d4 in baidu::zling::lz::ZlingRolzEncoder::Encode(unsigned char*, unsigned short*, int, int, int*) /home/nemequ/local/src/libzling/src/libzling_lz.cpp:143
    #3 0x7f81db48733e in baidu::zling::Encode(baidu::zling::Inputter*, baidu::zling::Outputter*, baidu::zling::ActionHandler*, int) /home/nemequ/local/src/libzling/src/libzling.cpp:203
    #4 0x40232c in main /home/nemequ/local/src/libzling/demo/zling.cpp:199
    #5 0x7f81da84257f in __libc_start_main (/lib64/libc.so.6+0x2057f)
    #6 0x402a58 in _start (/home/nemequ/local/src/libzling/build/zling_demo+0x402a58)

0x7f81d6cfcbbc is located 4016 bytes to the right of 10617868-byte region [0x7f81d62db800,0x7f81d6cfbc0c)
allocated by thread T0 here:
    #0 0x7f81db734872 in operator new(unsigned long) (/lib64/libasan.so.2+0x99872)
    #1 0x7f81db486946 in baidu::zling::EncodeResource::EncodeResource(int) /home/nemequ/local/src/libzling/src/libzling.cpp:118
    #2 0x7f81db486946 in baidu::zling::Encode(baidu::zling::Inputter*, baidu::zling::Outputter*, baidu::zling::ActionHandler*, int) /home/nemequ/local/src/libzling/src/libzling.cpp:179

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/nemequ/local/src/libzling/src/libzling_lz.cpp:266 baidu::zling::lz::ZlingRolzEncoder::MatchLazy(unsigned char*, int, int, int)
Shadow bytes around the buggy address:
  0x0ff0bad97920: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff0bad97930: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff0bad97940: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff0bad97950: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff0bad97960: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0ff0bad97970: fa fa fa fa fa fa fa[fa]fa fa fa fa fa fa fa fa
  0x0ff0bad97980: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff0bad97990: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff0bad979a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff0bad979b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff0bad979c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==27776==ABORTING

I don't usually bother fuzzing compressors, but after you fix this maybe I should try it with zling.

[nemequ]

I've been running AFL against libzling for ~ 15 hours (* 4 instances), no crashes. I think the issues in the decompressor are fixed.

[richox]

@nemequ the encoder bug is fixed.