search

FLIF-hub / FLIF

Free Lossless Image Format
Other
3.72k stars 229 forks source link

FLIF failed to allocate memory and crashes against some invali input #514

Open hongxuchen opened 6 years ago

hongxuchen commented 6 years ago

We found with our fuzzer one case that will trigger the "failed to allocate memory" error (and crashes with signal 11) which we think is an error. When compiled with Address Sanitizer, it reports:

==32399==ERROR: AddressSanitizer failed to allocate 0x7bc5782000 (531593961472) bytes of LargeMmapAllocator (error code: 12)
==32399==Process memory map follows:
    0x000000400000-0x000001047000   /home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif
    0x000001247000-0x00000124d000   /home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif
    0x00000124d000-0x00000125c000   /home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif
    0x00000125c000-0x000001f49000   
    0x00007fff7000-0x00008fff7000   
    0x00008fff7000-0x02008fff7000   
    0x02008fff7000-0x10007fff8000   
    0x600000000000-0x602000000000   
    0x602000000000-0x602000010000   
    0x602000010000-0x602e00000000   
    0x602e00000000-0x602e00010000   
    0x602e00010000-0x607000000000   
    0x607000000000-0x607000010000   
    0x607000010000-0x607e00000000   
    0x607e00000000-0x607e00010000   
    0x607e00010000-0x608000000000   
    0x608000000000-0x608000010000   
    0x608000010000-0x608e00000000   
    0x608e00000000-0x608e00010000   
    0x608e00010000-0x60f000000000   
    0x60f000000000-0x60f000010000   
    0x60f000010000-0x60fe00000000   
    0x60fe00000000-0x60fe00010000   
    0x60fe00010000-0x614000000000   
    0x614000000000-0x614000010000   
    0x614000010000-0x614e00000000   
    0x614e00000000-0x614e00010000   
    0x614e00010000-0x616000000000   
    0x616000000000-0x616000010000   
    0x616000010000-0x616e00000000   
    0x616e00000000-0x616e00010000   
    0x616e00010000-0x617000000000   
    0x617000000000-0x617000010000   
    0x617000010000-0x617e00000000   
    0x617e00000000-0x617e00010000   
    0x617e00010000-0x619000000000   
    0x619000000000-0x619000010000   
    0x619000010000-0x619e00000000   
    0x619e00000000-0x619e00010000   
    0x619e00010000-0x621000000000   
    0x621000000000-0x621000010000   
    0x621000010000-0x621e00000000   
    0x621e00000000-0x621e00010000   
    0x621e00010000-0x624000000000   
    0x624000000000-0x624000010000   
    0x624000010000-0x624e00000000   
    0x624e00000000-0x624e00010000   
    0x624e00010000-0x631000000000   
    0x631000000000-0x631000020000   
    0x631000020000-0x631e00000000   
    0x631e00000000-0x631e00010000   
    0x631e00010000-0x640000000000   
    0x640000000000-0x640000003000   
    0x7f3ad5200000-0x7f3ad5300000   
    0x7f3ad5400000-0x7f3ad5500000   
    0x7f3ad5600000-0x7f3ad5700000   
    0x7f3ad5800000-0x7f3ad5900000   
    0x7f3ad591f000-0x7f3ad7c71000   
    0x7f3ad7c71000-0x7f3ad7e58000   /lib/x86_64-linux-gnu/libc-2.27.so
    0x7f3ad7e58000-0x7f3ad8058000   /lib/x86_64-linux-gnu/libc-2.27.so
    0x7f3ad8058000-0x7f3ad805c000   /lib/x86_64-linux-gnu/libc-2.27.so
    0x7f3ad805c000-0x7f3ad805e000   /lib/x86_64-linux-gnu/libc-2.27.so
    0x7f3ad805e000-0x7f3ad8062000   
    0x7f3ad8062000-0x7f3ad8079000   /lib/x86_64-linux-gnu/libgcc_s.so.1
    0x7f3ad8079000-0x7f3ad8278000   /lib/x86_64-linux-gnu/libgcc_s.so.1
    0x7f3ad8278000-0x7f3ad8279000   /lib/x86_64-linux-gnu/libgcc_s.so.1
    0x7f3ad8279000-0x7f3ad827a000   /lib/x86_64-linux-gnu/libgcc_s.so.1
    0x7f3ad827a000-0x7f3ad827d000   /lib/x86_64-linux-gnu/libdl-2.27.so
    0x7f3ad827d000-0x7f3ad847c000   /lib/x86_64-linux-gnu/libdl-2.27.so
    0x7f3ad847c000-0x7f3ad847d000   /lib/x86_64-linux-gnu/libdl-2.27.so
    0x7f3ad847d000-0x7f3ad847e000   /lib/x86_64-linux-gnu/libdl-2.27.so
    0x7f3ad847e000-0x7f3ad8485000   /lib/x86_64-linux-gnu/librt-2.27.so
    0x7f3ad8485000-0x7f3ad8684000   /lib/x86_64-linux-gnu/librt-2.27.so
    0x7f3ad8684000-0x7f3ad8685000   /lib/x86_64-linux-gnu/librt-2.27.so
    0x7f3ad8685000-0x7f3ad8686000   /lib/x86_64-linux-gnu/librt-2.27.so
    0x7f3ad8686000-0x7f3ad86a0000   /lib/x86_64-linux-gnu/libpthread-2.27.so
    0x7f3ad86a0000-0x7f3ad889f000   /lib/x86_64-linux-gnu/libpthread-2.27.so
    0x7f3ad889f000-0x7f3ad88a0000   /lib/x86_64-linux-gnu/libpthread-2.27.so
    0x7f3ad88a0000-0x7f3ad88a1000   /lib/x86_64-linux-gnu/libpthread-2.27.so
    0x7f3ad88a1000-0x7f3ad88a5000   
    0x7f3ad88a5000-0x7f3ad8a42000   /lib/x86_64-linux-gnu/libm-2.27.so
    0x7f3ad8a42000-0x7f3ad8c41000   /lib/x86_64-linux-gnu/libm-2.27.so
    0x7f3ad8c41000-0x7f3ad8c42000   /lib/x86_64-linux-gnu/libm-2.27.so
    0x7f3ad8c42000-0x7f3ad8c43000   /lib/x86_64-linux-gnu/libm-2.27.so
    0x7f3ad8c43000-0x7f3ad8dbc000   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
    0x7f3ad8dbc000-0x7f3ad8fbc000   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
    0x7f3ad8fbc000-0x7f3ad8fc6000   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
    0x7f3ad8fc6000-0x7f3ad8fc8000   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
    0x7f3ad8fc8000-0x7f3ad8fcc000   
    0x7f3ad8fcc000-0x7f3ad8fe8000   /lib/x86_64-linux-gnu/libz.so.1.2.11
    0x7f3ad8fe8000-0x7f3ad91e7000   /lib/x86_64-linux-gnu/libz.so.1.2.11
    0x7f3ad91e7000-0x7f3ad91e8000   /lib/x86_64-linux-gnu/libz.so.1.2.11
    0x7f3ad91e8000-0x7f3ad91e9000   /lib/x86_64-linux-gnu/libz.so.1.2.11
    0x7f3ad91e9000-0x7f3ad921a000   /usr/lib/x86_64-linux-gnu/libpng16.so.16.34.0
    0x7f3ad921a000-0x7f3ad9419000   /usr/lib/x86_64-linux-gnu/libpng16.so.16.34.0
    0x7f3ad9419000-0x7f3ad941a000   /usr/lib/x86_64-linux-gnu/libpng16.so.16.34.0
    0x7f3ad941a000-0x7f3ad941b000   /usr/lib/x86_64-linux-gnu/libpng16.so.16.34.0
    0x7f3ad941b000-0x7f3ad9421000   /usr/lib/x86_64-linux-gnu/libgtk3-nocsd.so.0
    0x7f3ad9421000-0x7f3ad9620000   /usr/lib/x86_64-linux-gnu/libgtk3-nocsd.so.0
    0x7f3ad9620000-0x7f3ad9621000   /usr/lib/x86_64-linux-gnu/libgtk3-nocsd.so.0
    0x7f3ad9621000-0x7f3ad9622000   /usr/lib/x86_64-linux-gnu/libgtk3-nocsd.so.0
    0x7f3ad9622000-0x7f3ad9649000   /lib/x86_64-linux-gnu/ld-2.27.so
    0x7f3ad96ac000-0x7f3ad97f9000   
    0x7f3ad97f9000-0x7f3ad9844000   
    0x7f3ad9844000-0x7f3ad9847000   [vvar]
    0x7f3ad9847000-0x7f3ad9849000   [vdso]
    0x7f3ad9849000-0x7f3ad984a000   /lib/x86_64-linux-gnu/ld-2.27.so
    0x7f3ad984a000-0x7f3ad984b000   /lib/x86_64-linux-gnu/ld-2.27.so
    0x7f3ad984b000-0x7f3ad984c000   
    0x7fff645d4000-0x7fff645f8000   [stack]
    0xffffffffff600000-0xffffffffff601000   [vsyscall]
==32399==End of process memory map.
==32399==AddressSanitizer CHECK failed: /build/llvm-toolchain-4.0-iMkWTm/llvm-toolchain-4.0-4.0.1/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:120 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
    #0 0x4e7b55 in __asan::AsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif+0x4e7b55)
    #1 0x5036f5 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif+0x5036f5)
    #2 0x4f2492 in __sanitizer::ReportMmapFailureAndDie(unsigned long, char const*, char const*, int, bool) (/home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif+0x4f2492)
    #3 0x4fc915 in __sanitizer::MmapOrDie(unsigned long, char const*, bool) (/home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif+0x4fc915)
    #4 0x4348a1 in __asan::Allocator::Allocate(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) (/home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif+0x4348a1)
    #5 0x51884f in operator new(unsigned long) (/home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif+0x51884f)
    #6 0x5542ef in __gnu_cxx::new_allocator<unsigned char>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/8.1.0/../../../../include/c++/8.1.0/ext/new_allocator.h:111:27
    #7 0x5542ef in std::allocator_traits<std::allocator<unsigned char> >::allocate(std::allocator<unsigned char>&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/8.1.0/../../../../include/c++/8.1.0/bits/alloc_traits.h:436
    #8 0x5542ef in std::_Vector_base<unsigned char, std::allocator<unsigned char> >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/8.1.0/../../../../include/c++/8.1.0/bits/stl_vector.h:296
    #9 0x5542ef in std::_Vector_base<unsigned char, std::allocator<unsigned char> >::_M_create_storage(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/8.1.0/../../../../include/c++/8.1.0/bits/stl_vector.h:311
    #10 0x5542ef in std::_Vector_base<unsigned char, std::allocator<unsigned char> >::_Vector_base(unsigned long, std::allocator<unsigned char> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/8.1.0/../../../../include/c++/8.1.0/bits/stl_vector.h:260
    #11 0x5542ef in std::vector<unsigned char, std::allocator<unsigned char> >::vector(unsigned long, unsigned char const&, std::allocator<unsigned char> const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/8.1.0/../../../../include/c++/8.1.0/bits/stl_vector.h:429
    #12 0x5542ef in Plane<unsigned char>::Plane(unsigned long, unsigned long, int, int) /home/hongxu/FOT/FLIF-fuzz/src/image/image.hpp:231
    #13 0x54a667 in std::unique_ptr<Plane<unsigned char>, std::default_delete<Plane<unsigned char> > > make_unique<Plane<unsigned char>, unsigned long&, unsigned long&, int, int&>(unsigned long&, unsigned long&, int&&, int&) /home/hongxu/FOT/FLIF-fuzz/src/image/image.hpp:159:35
    #14 0x54a667 in Image::real_init(bool) /home/hongxu/FOT/FLIF-fuzz/src/image/image.hpp:728
    #15 0x56082b in Image::init(unsigned int, unsigned int, int, int, int) /home/hongxu/FOT/FLIF-fuzz/src/image/image.hpp:687:14
    #16 0x56082b in image_load_pnm(char const*, Image&) /home/hongxu/FOT/FLIF-fuzz/src/image/image-pnm.cpp:75
    #17 0x53d3c7 in Image::load(char const*, metadata_options&) /home/hongxu/FOT/FLIF-fuzz/src/image/image.cpp:54:9
    #18 0x51def1 in encode_load_input_images(int, char**, std::vector<Image, std::allocator<Image> >&, flif_options&) /home/hongxu/FOT/FLIF-fuzz/src/flif.cpp:230:20
    #19 0x531307 in handle_encode(int, char**, std::vector<Image, std::allocator<Image> >&, flif_options&) /home/hongxu/FOT/FLIF-fuzz/src/flif.cpp:356:10
    #20 0x531307 in main /home/hongxu/FOT/FLIF-fuzz/src/flif.cpp:763
    #21 0x7f3ad7c92b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
    #22 0x429319 in _start (/home/hongxu/FOT/FLIF-fuzz/install/usr/local/bin/flif+0x429319)

PoC file: allocate_01.txt allocate_02.txt

fgeek commented 6 years ago

This is normal when you are fuzzing with ASan. Usually these are non-issues.

hongxuchen commented 6 years ago

@fgeek It crashes with normal build on my machine.

fgeek commented 6 years ago

@HongxuChen Do you get segfault or out of memory error? I can test this also a bit later.

hongxuchen commented 6 years ago

@fgeek FYI, here is the screenshot for the normal build without asan. 2018-06-13-132631_1919x229_scrot