search

jasper-software / jasper

Official Repository for the JasPer Image Coding Toolkit
http://www.ece.uvic.ca/~mdadams/jasper
Other
226 stars 101 forks source link

double-free in mem_close (jas_stream.c) #25

Closed asarubbo closed 8 years ago

asarubbo commented 8 years ago

Hello, time ago I found multiple crashes on jasper. I didn't know where to post the bugs since the development seems dead, so I just informed the community on oss-security. Now I discovered that the development is still active, here the details:

# imginfo $FILE
Corrupt JPEG data: 1 extraneous bytes before marker 0xc4                                                                                                                                                                                                                       
=================================================================                                                                                                                                                                                                              
==9405==ERROR: AddressSanitizer: attempting double-free on 0x619000003780 in thread T0:                                                                                                                                                                                        
    #0 0x4bfe10 in __interceptor_free /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:38                                                                                                                       
    #1 0x7fe9caf2e160 in mem_close /tmp/jasper-version-1.900.3/src/libjasper/base/jas_stream.c:1079:3                                                                                                                                                                          
    #2 0x7fe9caf28cdb in jas_stream_close /tmp/jasper-version-1.900.3/src/libjasper/base/jas_stream.c:466:2                                                                                                                                                                    
    #3 0x7fe9caefc981 in jas_image_cmpt_destroy /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:343:3                                                                                                                                                               
    #4 0x7fe9caefc981 in jas_image_cmpt_create /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:333                                                                                                                                                                  
    #5 0x7fe9caeff787 in jas_image_addcmpt /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:677:18                                                                                                                                                                   
    #6 0x7fe9cafc49ac in jpg_mkimage /tmp/jasper-version-1.900.3/src/libjasper/jpg/jpg_dec.c:247:7                                                                                                                                                                             
    #7 0x7fe9cafc49ac in jpg_decode /tmp/jasper-version-1.900.3/src/libjasper/jpg/jpg_dec.c:171                                                                                                                                                                                
    #8 0x7fe9caefda9a in jas_image_decode /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:372:16                                                                                                                                                                    
    #9 0x4f11bd in main /tmp/jasper-version-1.900.3/src/appl/imginfo.c:179:16                                                                                                                                                                                                  
    #10 0x7fe9ca01561f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289                                                                                                                                                       
    #11 0x418bc8 in _start (/tmp/jasper-version-1.900.3/src/appl/.libs/imginfo+0x418bc8)                                                                                                                                                                                       

0x619000003780 is located 0 bytes inside of 1024-byte region [0x619000003780,0x619000003b80)                                                                                                                                                                                   
freed by thread T0 here:                                                                                                                                                                                                                                                       
    #0 0x4c0498 in realloc /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:71                                                                                                                                  
    #1 0x7fe9caf2dd53 in mem_resize /tmp/jasper-version-1.900.3/src/libjasper/base/jas_stream.c:995:14                                                                                                                                                                         
    #2 0x7fe9caf2dd53 in mem_write /tmp/jasper-version-1.900.3/src/libjasper/base/jas_stream.c:1018                                                                                                                                                                            
    #3 0x7fe9caf2b0b3 in jas_stream_flushbuf /tmp/jasper-version-1.900.3/src/libjasper/base/jas_stream.c:819:7                                                                                                                                                                 
    #4 0x7fe9caf2cb14 in jas_stream_flush /tmp/jasper-version-1.900.3/src/libjasper/base/jas_stream.c:749:9                                                                                                                                                                    
    #5 0x7fe9caf2cb14 in jas_stream_seek /tmp/jasper-version-1.900.3/src/libjasper/base/jas_stream.c:656                                                                                                                                                                       
    #6 0x7fe9caefc95a in jas_image_cmpt_create /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:332:4                                                                                                                                                                
    #7 0x7fe9caeff787 in jas_image_addcmpt /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:677:18                                                                                                                                                                   
    #8 0x7fe9cafc49ac in jpg_mkimage /tmp/jasper-version-1.900.3/src/libjasper/jpg/jpg_dec.c:247:7                                                                                                                                                                             
    #9 0x7fe9cafc49ac in jpg_decode /tmp/jasper-version-1.900.3/src/libjasper/jpg/jpg_dec.c:171                                                                                                                                                                                
    #10 0x7fe9caefda9a in jas_image_decode /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:372:16                                                                                                                                                                   
    #11 0x4f11bd in main /tmp/jasper-version-1.900.3/src/appl/imginfo.c:179:16                                                                                                                                                                                                 
    #12 0x7fe9ca01561f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289                                                                                                                                                       

previously allocated by thread T0 here:                                                                                                                                                                                                                                        
    #0 0x4c0118 in malloc /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:52                                                                                                                                   
    #1 0x7fe9caf2885e in jas_stream_memopen /tmp/jasper-version-1.900.3/src/libjasper/base/jas_stream.c:215:15                                                                                                                                                                 
    #2 0x7fe9caefc78e in jas_image_cmpt_create /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:322:28                                                                                                                                                               
    #3 0x7fe9caeff787 in jas_image_addcmpt /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:677:18                                                                                                                                                                   
    #4 0x7fe9cafc49ac in jpg_mkimage /tmp/jasper-version-1.900.3/src/libjasper/jpg/jpg_dec.c:247:7                                                                                                                                                                             
    #5 0x7fe9cafc49ac in jpg_decode /tmp/jasper-version-1.900.3/src/libjasper/jpg/jpg_dec.c:171                                                                                                                                                                                
    #6 0x7fe9caefda9a in jas_image_decode /tmp/jasper-version-1.900.3/src/libjasper/base/jas_image.c:372:16                                                                                                                                                                    
    #7 0x4f11bd in main /tmp/jasper-version-1.900.3/src/appl/imginfo.c:179:16                                                                                                                                                                                                  
    #8 0x7fe9ca01561f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289                                                                                                                                                        

SUMMARY: AddressSanitizer: double-free /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:38 in __interceptor_free
==9405==ABORTING

Tested against the latest 1.900.3 Testcase: 1.crash.zip

mdadams commented 8 years ago

I cannot seem to reproduce the above problem of a double free with the versions of GCC and Clang that I have. I did get some USAN warnings about undefined behavior on shift operations in the MIF and RAS codecs, however. The MIF and RAS code is executing due to auto-detection of the file format. I fixed this USAN problem, which was due to unsigned chars being promoted to int instead of being cast to the correct unsigned type. Maybe some of the other changes that I have committed to the master branch since the 1.900.3 release have fixed your problem? I have tagged a new release version-1.900.4 in the repo. Can you try the above test case again in your environment with version-1.900.4 and let me know if the bug is fixed or still there? Unless I hear otherwise, I will assume that this bug is fixed.

asarubbo commented 8 years ago

I still can reproduce.

Stacktrace for 1.900.4:

Corrupt JPEG data: 1 extraneous bytes before marker 0xc4                                                                                                                                       
=================================================================                                                                                                                              
==31536==ERROR: AddressSanitizer: attempting double-free on 0x619000003780 in thread T0:                                                                                                       
    #0 0x4bfe10 in __interceptor_free /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:38                                       
    #1 0x7f15e7385450 in mem_close /tmp/jasper-version-1.900.4/src/libjasper/base/jas_stream.c:1079:3                                                                                          
    #2 0x7f15e737ffcb in jas_stream_close /tmp/jasper-version-1.900.4/src/libjasper/base/jas_stream.c:466:2                                                                                    
    #3 0x7f15e7353b71 in jas_image_cmpt_destroy /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:343:3                                                                               
    #4 0x7f15e7353b71 in jas_image_cmpt_create /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:333                                                                                  
    #5 0x7f15e7356977 in jas_image_addcmpt /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:677:18                                                                                   
    #6 0x7f15e741bd7c in jpg_mkimage /tmp/jasper-version-1.900.4/src/libjasper/jpg/jpg_dec.c:247:7                                                                                             
    #7 0x7f15e741bd7c in jpg_decode /tmp/jasper-version-1.900.4/src/libjasper/jpg/jpg_dec.c:171                                                                                                
    #8 0x7f15e7354c8a in jas_image_decode /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:372:16                                                                                    
    #9 0x4f11bd in main /tmp/jasper-version-1.900.4/src/appl/imginfo.c:179:16                                                                                                                  
    #10 0x7f15e646c61f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289                                                                       
    #11 0x418bc8 in _start (/tmp/jasper-version-1.900.4/src/appl/.libs/imginfo+0x418bc8)                                                                                                       

0x619000003780 is located 0 bytes inside of 1024-byte region [0x619000003780,0x619000003b80)                                                                                                   
freed by thread T0 here:                                                                                                                                                                       
    #0 0x4c0498 in realloc /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:71                                                  
    #1 0x7f15e7385048 in mem_resize /tmp/jasper-version-1.900.4/src/libjasper/base/jas_stream.c:995:14                                                                                         
    #2 0x7f15e7385048 in mem_write /tmp/jasper-version-1.900.4/src/libjasper/base/jas_stream.c:1018                                                                                            
    #3 0x7f15e73823a3 in jas_stream_flushbuf /tmp/jasper-version-1.900.4/src/libjasper/base/jas_stream.c:819:7                                                                                 
    #4 0x7f15e7383e04 in jas_stream_flush /tmp/jasper-version-1.900.4/src/libjasper/base/jas_stream.c:749:9                                                                                    
    #5 0x7f15e7383e04 in jas_stream_seek /tmp/jasper-version-1.900.4/src/libjasper/base/jas_stream.c:656                                                                                       
    #6 0x7f15e7353b4a in jas_image_cmpt_create /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:332:4                                                                                
    #7 0x7f15e7356977 in jas_image_addcmpt /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:677:18                                                                                   
    #8 0x7f15e741bd7c in jpg_mkimage /tmp/jasper-version-1.900.4/src/libjasper/jpg/jpg_dec.c:247:7                                                                                             
    #9 0x7f15e741bd7c in jpg_decode /tmp/jasper-version-1.900.4/src/libjasper/jpg/jpg_dec.c:171                                                                                                
    #10 0x7f15e7354c8a in jas_image_decode /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:372:16                                                                                   
    #11 0x4f11bd in main /tmp/jasper-version-1.900.4/src/appl/imginfo.c:179:16
    #12 0x7f15e646c61f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289

previously allocated by thread T0 here:
    #0 0x4c0118 in malloc /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:52
    #1 0x7f15e737fb4e in jas_stream_memopen /tmp/jasper-version-1.900.4/src/libjasper/base/jas_stream.c:215:15
    #2 0x7f15e735397e in jas_image_cmpt_create /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:322:28
    #3 0x7f15e7356977 in jas_image_addcmpt /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:677:18
    #4 0x7f15e741bd7c in jpg_mkimage /tmp/jasper-version-1.900.4/src/libjasper/jpg/jpg_dec.c:247:7
    #5 0x7f15e741bd7c in jpg_decode /tmp/jasper-version-1.900.4/src/libjasper/jpg/jpg_dec.c:171
    #6 0x7f15e7354c8a in jas_image_decode /tmp/jasper-version-1.900.4/src/libjasper/base/jas_image.c:372:16
    #7 0x4f11bd in main /tmp/jasper-version-1.900.4/src/appl/imginfo.c:179:16
    #8 0x7f15e646c61f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289

SUMMARY: AddressSanitizer: double-free /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:38 in __interceptor_free
==31536==ABORTING
mdadams commented 8 years ago

I am sorry but I still cannot reproduce this problem. I reviewed the code involved in the double free from your trace information above and I cannot see how it can do a double free, unless there is some sort of memory corruption or other bug that is the real problem. Of course, I could be overlooking something. In any case, it is very difficult to try to resolve the problem when I cannot reproduce it myself. The fact that I cannot reproduce the problem on my machine might be an indication that the double free is not itself the real problem but rather a symptom of something else, such as a memory corruption bug.

asarubbo commented 8 years ago

if it can work, I can share my build. But apart says that I'm compiling it with clang, there is nothing more....

Which compiler are you using? version?

mdadams commented 8 years ago

The Linux distro that I am using comes with GCC 4.9.2 and Clang 3.5.0 out of the box. I have manually installed GCC 6.1, however, and have been using this mostly. It seems that a lot of sanitizer functionality is absent/flakey in the older compiler versions that I have (at least for C++). GCC 6.1 seems okay as far as sanitizers go, however. Can you reproduce the problem when you build as follows with Clang (or even better with GCC 6.1): [checkout fresh copy of jasper version-1.900.6 code] autoreconf -i ./configure --prefix=/tmp/jasper --enable-shared=no --enable-debug=yes make clean && make The --enable-debug option should cause various sanitizer flags to be used for compiling (and as far as I can tell no special linker flags are needed with GCC for sanitizers unless static linking with sanitizer libraries is needed). I think that the same compiler/linker flags should work with Clang (since that is what you are using).

Suppose that I build jasper in the above way and then run the following: src/appl/imginfo < 1.crash In this case, I get the following output: Corrupt JPEG data: 1 extraneous bytes before marker 0xc4 Huffman table 0x01 was not defined The exit status is 1. Do you still get a double-free ASAN failure when you build in the above manner? There is no ASAN failure.

asarubbo commented 8 years ago

Yes, I can reproduce in the same way you described.

wget https://github.com/mdadams/jasper/archive/version-1.900.6.tar.gz
tar xzf version-1.900.6.tar.gz 
cd jasper-version-1.900.6/
autoreconf -i
./configure --prefix=/tmp/jasper --enable-shared=no --enable-debug=yes
make clean
make
cd /tmp
wget https://github.com/mdadams/jasper/files/527267/1.crash.zip
unzip 1.crash.zip 
cd jasper-version-1.900.6/
./src/appl/imginfo < /tmp/1.crash 
Corrupt JPEG data: 1 extraneous bytes before marker 0xc4
=================================================================
==11226==ERROR: AddressSanitizer: attempting double-free on 0x619000003780 in thread T0:
    #0 0x7f6c1c0b84e7 (/usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libasan.so.1.0.0+0x544e7)
    #1 0x41370b (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x41370b)
    #2 0x4212b7 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x4212b7)
    #3 0x41a798 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x41a798)
    #4 0x405e84 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x405e84)
    #5 0x405dd1 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x405dd1)
    #6 0x408f36 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x408f36)
    #7 0x4ea459 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x4ea459)
    #8 0x4e9afb (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x4e9afb)
    #9 0x405f88 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x405f88)
    #10 0x402ad4 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x402ad4)
    #11 0x7f6c1aab661f (/lib64/libc-2.22.so+0x2061f)
    #12 0x402758 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x402758)

0x619000003780 is located 0 bytes inside of 1024-byte region [0x619000003780,0x619000003b80)
freed by thread T0 here:
    #0 0x7f6c1c0b89b6 (/usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libasan.so.1.0.0+0x549b6)
    #1 0x413a48 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x413a48)
    #2 0x420533 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x420533)
    #3 0x42080a (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x42080a)
    #4 0x41e6ae (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x41e6ae)
    #5 0x41d7a4 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x41d7a4)
    #6 0x41cab2 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x41cab2)
    #7 0x405dc0 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x405dc0)
    #8 0x408f36 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x408f36)
    #9 0x4ea459 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x4ea459)
    #10 0x4e9afb (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x4e9afb)
    #11 0x405f88 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x405f88)
    #12 0x402ad4 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x402ad4)
    #13 0x7f6c1aab661f (/lib64/libc-2.22.so+0x2061f)

previously allocated by thread T0 here:
    #0 0x7f6c1c0b86ff (/usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libasan.so.1.0.0+0x546ff)
    #1 0x4136cc (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x4136cc)
    #2 0x41924a (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x41924a)
    #3 0x405be7 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x405be7)
    #4 0x408f36 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x408f36)
    #5 0x4ea459 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x4ea459)
    #6 0x4e9afb (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x4e9afb)
    #7 0x405f88 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x405f88)
    #8 0x402ad4 (/tmp/jasper-version-1.900.6/src/appl/imginfo+0x402ad4)
    #9 0x7f6c1aab661f (/lib64/libc-2.22.so+0x2061f)

SUMMARY: AddressSanitizer: double-free ??:0 ??
==11226==ABORTING
mdadams commented 8 years ago

It looks like you were using GCC 4.9.3 in the above case. Is the correct? If so, it is strange that you can reproduce the problem with both GCC 4.9.3 and Clang, but I cannot. :(

asarubbo commented 8 years ago

Yes, it is gcc-4.9.3 used to reproduce your sequence of commands. usually I'm using clang.

Can I help in something else? btw, since the duplicate of this, as #31 I'm not the only able to reproduce.

mdadams commented 8 years ago

I tried this test case on two other systems. I still cannot reproduce the problem. All of the systems that I have access to are running variants of RHEL/Fedora/CentOS. So, maybe the problem does not manifest itself on these systems? This double free problem and #31 are for JPEG data and this uses a JPEG library that is not part of JasPer. Which version of this JPEG library are you using? Is this problem happening only for JPEG data? (Only JPEG 2000 is handlled directly by JasPer. JPEG is handed off to another library.) It seems that I am using libjpeg-turbo 1.3.1-5, on my main development machine. I am wondering if the JPEG code from this other library might be stomping on memory.

asarubbo commented 8 years ago

There is no problem, let me check tomorrow if I can reproduce on a fresh gentoo chroot. Then I can give you the chroot tarball or post the instructions to reproduce. Would you mind to reopen in the meantime?

jridky commented 8 years ago

Bad news unfortunately. I am too able to reproduce this issue on my Fedora 24 with gcc-6.1.1, clang-2.8.0 and libjpeg-turbo 1.5.0.

Valgrind output:

valgrind imginfo -f /tmp/1.crash

Corrupt JPEG data: 1 extraneous bytes before marker 0xc4
==29047== Invalid free() / delete / delete[] / realloc()
==29047==    at 0x4C2CD5A: free (vg_replace_malloc.c:530)
==29047==    by 0x4E4CC57: ??? (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E4DC33: jas_stream_close (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E45A01: ??? (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E45B5F: ??? (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E46A90: jas_image_addcmpt (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E71813: jpg_decode (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E46BCC: jas_image_decode (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x108D67: ??? (in /usr/bin/imginfo)
==29047==    by 0x5621730: (below main) (libc-start.c:289)
==29047==  Address 0x5bed5b0 is 0 bytes inside a block of size 1,024 free'd
==29047==    at 0x4C2CCC0: free (vg_replace_malloc.c:529)
==29047==    by 0x4C2DCAE: realloc (vg_replace_malloc.c:785)
==29047==    by 0x4E4CE8D: ??? (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E4D76E: jas_stream_flushbuf (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E4DD9C: jas_stream_seek (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E45B1E: ??? (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E46A90: jas_image_addcmpt (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E71813: jpg_decode (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E46BCC: jas_image_decode (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x108D67: ??? (in /usr/bin/imginfo)
==29047==    by 0x5621730: (below main) (libc-start.c:289)
==29047==  Block was alloc'd at
==29047==    at 0x4C2BBAD: malloc (vg_replace_malloc.c:299)
==29047==    by 0x4E4DCF8: jas_stream_memopen (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E45B48: ??? (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E46A90: jas_image_addcmpt (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E71813: jpg_decode (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x4E46BCC: jas_image_decode (in /usr/lib64/libjasper.so.1.0.0)
==29047==    by 0x108D67: ??? (in /usr/bin/imginfo)
==29047==    by 0x5621730: (below main) (libc-start.c:289)
==29047== 
cannot load image
mdadams commented 8 years ago

Unfortunately, I cannot reproduce the problem with ASAN or valgrind. I did find a place in the JPEG decode logic where a double fclose of a stream can happen, by a visual inspection of the code. I have fixed this problem. This bug would appear unrelated to the one that you are seeing, but it is hard to say for certain since this bug could corrupt the heap and it would result in a double free, but for a different chunk of memory. By any chance did my change fix your issue? The relevant commit with my bug fix is:

commit 668e68295306a78a5e428df2ed80027c4eece964 Author: Michael Adams mdadams@ece.uvic.ca Date: Tue Oct 18 08:11:07 2016 -0700

Fix a potential double fclose of a FILE* in the JPEG decoder.

P.S. I have re-opened this issue, since it is still unresolved (unless my change fixed things).

asarubbo commented 8 years ago

I still can reproduce. I'll give you a way to reproduce it; maybe a chroot or something similar.

mdadams commented 8 years ago

When I run your test case, the jpg_decode function calls jpeg_start_decompress (in the JPEG library), which exits the program. (The JPEG library exits upon error.) When you run the test case, it seems that the JPEG library does not exit in jpeg_start_decompress. Instead, it continues and later causes a double free (maybe in jpg_mkimage?).

I added some extra debugging prints in the memory allocator code as well as the JPEG decoder. If you take the latest version of the code from the master branch, and then run imginfo with the --debug-level 100 option, you will generate a lot of additional output. Some of this cutput might be helpful to me in pinpointing the problem.

hannob commented 8 years ago

debug info (from imginfo --debug-level 100):

px jasper # src/appl/imginfo --debug-level 100 < 1.crash 
jas_malloc(104) -> 0x60b00000af90
jas_malloc(32) -> 0x60300000ee60
jas_malloc(8208) -> 0x62500000c900
Corrupt JPEG data: 1 extraneous bytes before marker 0xc4
header: image_width 65279; image_height 65343; num_components 3
header: output_width 65279; output_height 65343; output_components 3
jas_malloc(72) -> 0x60700000dfb0
jas_malloc(1024) -> 0x619000003c80
jas_malloc(80) -> 0x60700000df40
jas_malloc(104) -> 0x60b00000aee0
jas_malloc(8208) -> 0x62500000a100
jas_malloc(40) -> 0x60400000df90
jas_malloc(1024) -> 0x619000003780
jas_realloc(0x619000003780, 0) -> (nil)
jas_free(0x619000003780)
=================================================================
==7609==ERROR: AddressSanitizer: attempting double-free on 0x619000003780 in thread T0:
    #0 0x4c0f00 in __interceptor_free (/f/jasper/jasper/src/appl/imginfo+0x4c0f00)
    #1 0x510dfd in mem_close /f/jasper/jasper/src/libjasper/base/jas_stream.c:1084:3
    #2 0x507e57 in jas_stream_close /f/jasper/jasper/src/libjasper/base/jas_stream.c:466:2
    #3 0x4f4823 in jas_image_cmpt_destroy /f/jasper/jasper/src/libjasper/base/jas_image.c:362:3
    #4 0x4f4823 in jas_image_cmpt_create /f/jasper/jasper/src/libjasper/base/jas_image.c:354
    #5 0x4f93e8 in jas_image_addcmpt /f/jasper/jasper/src/libjasper/base/jas_image.c:697:18
    #6 0x5b67c2 in jpg_mkimage /f/jasper/jasper/src/libjasper/jpg/jpg_dec.c:268:7
    #7 0x5b67c2 in jpg_decode /f/jasper/jasper/src/libjasper/jpg/jpg_dec.c:183
    #8 0x4f6042 in jas_image_decode /f/jasper/jasper/src/libjasper/base/jas_image.c:391:16
    #9 0x4f23cf in main /f/jasper/jasper/src/appl/imginfo.c:188:16
    #10 0x7f43cb7a978f in __libc_start_main (/lib64/libc.so.6+0x2078f)
    #11 0x4195d8 in _start (/f/jasper/jasper/src/appl/imginfo+0x4195d8)

0x619000003780 is located 0 bytes inside of 1024-byte region [0x619000003780,0x619000003b80)
freed by thread T0 here:
    #0 0x4c1588 in realloc (/f/jasper/jasper/src/appl/imginfo+0x4c1588)
    #1 0x502011 in jas_realloc /f/jasper/jasper/src/libjasper/base/jas_malloc.c:114:11
    #2 0x502011 in jas_realloc2 /f/jasper/jasper/src/libjasper/base/jas_malloc.c:155

previously allocated by thread T0 here:
    #0 0x4c1208 in malloc (/f/jasper/jasper/src/appl/imginfo+0x4c1208)
    #1 0x501a50 in jas_malloc /f/jasper/jasper/src/libjasper/base/jas_malloc.c:105:11

SUMMARY: AddressSanitizer: double-free (/f/jasper/jasper/src/appl/imginfo+0x4c0f00) in __interceptor_free
==7609==ABORTING
jridky commented 8 years ago

I am able to confirm previous comment:

valgrind imginfo --debug-level 100 -f /tmp/1.crash 

==2341== Memcheck, a memory error detector
==2341== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==2341== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==2341== Command: imginfo --debug-level 100 -f /tmp/1.crash
==2341== 
jas_malloc(104) -> 0x59c4a70
jas_malloc(32) -> 0x59c4b20
jas_malloc(8208) -> 0x59c4b80
Corrupt JPEG data: 1 extraneous bytes before marker 0xc4
header: image_width 65279; image_height 65343; num_components 3
header: output_width 65279; output_height 65343; output_components 3
jas_malloc(72) -> 0x5bebee0
jas_malloc(1024) -> 0x5bebf70
jas_malloc(80) -> 0x5bec3b0
jas_malloc(104) -> 0x5bec440
jas_malloc(8208) -> 0x5bec4f0
jas_malloc(40) -> 0x5bee540
jas_malloc(1024) -> 0x5bee5b0
jas_realloc(0x5bee5b0, 0) -> (nil)
jas_free(0x5bee5b0)
==2341== Invalid free() / delete / delete[] / realloc()
==2341==    at 0x4C2CD5A: free (vg_replace_malloc.c:530)
==2341==    by 0x4E4CF77: mem_close (jas_stream.c:1084)
==2341==    by 0x4E4E013: jas_stream_close (jas_stream.c:466)
==2341==    by 0x4E459E1: jas_image_cmpt_destroy (jas_image.c:362)
==2341==    by 0x4E45AB8: jas_image_cmpt_create (jas_image.c:354)
==2341==    by 0x4E46AC0: jas_image_addcmpt (jas_image.c:697)
==2341==    by 0x4E72043: jpg_mkimage (jpg_dec.c:268)
==2341==    by 0x4E72043: jpg_decode (jpg_dec.c:183)
==2341==    by 0x4E46BFC: jas_image_decode (jas_image.c:391)
==2341==    by 0x108E30: main (imginfo.c:188)
==2341==  Address 0x5bee5b0 is 0 bytes inside a block of size 1,024 free'd
==2341==    at 0x4C2CCC0: free (vg_replace_malloc.c:529)
==2341==    by 0x4C2DCAE: realloc (vg_replace_malloc.c:785)
==2341==    by 0x4E4BF5E: jas_realloc (jas_malloc.c:114)
==2341==    by 0x4E4D214: mem_resize (jas_stream.c:998)
==2341==    by 0x4E4D214: mem_write (jas_stream.c:1023)
==2341==    by 0x4E4DB4E: jas_stream_flushbuf (jas_stream.c:819)
==2341==    by 0x4E4E17C: jas_stream_seek (jas_stream.c:656)
==2341==    by 0x4E45B62: jas_image_cmpt_create (jas_image.c:346)
==2341==    by 0x4E46AC0: jas_image_addcmpt (jas_image.c:697)
==2341==    by 0x4E72043: jpg_mkimage (jpg_dec.c:268)
==2341==    by 0x4E72043: jpg_decode (jpg_dec.c:183)
==2341==    by 0x4E46BFC: jas_image_decode (jas_image.c:391)
==2341==    by 0x108E30: main (imginfo.c:188)
==2341==  Block was alloc'd at
==2341==    at 0x4C2BBAD: malloc (vg_replace_malloc.c:299)
==2341==    by 0x4E4BEEA: jas_malloc (jas_malloc.c:105)
==2341==    by 0x4E4E0D8: jas_stream_memopen (jas_stream.c:215)
==2341==    by 0x4E45B30: jas_image_cmpt_create (jas_image.c:335)
==2341==    by 0x4E46AC0: jas_image_addcmpt (jas_image.c:697)
==2341==    by 0x4E72043: jpg_mkimage (jpg_dec.c:268)
==2341==    by 0x4E72043: jpg_decode (jpg_dec.c:183)
==2341==    by 0x4E46BFC: jas_image_decode (jas_image.c:391)
==2341==    by 0x108E30: main (imginfo.c:188)
==2341== 
jas_free(0x5bee540)
jas_free(0x5bec4f0)
jas_free(0x5bec440)
jas_free(0x5bec3b0)
jas_free(0x5bebf70)
jas_free(0x5bebee0)
jpg_mkimage failed
jas_free(0x59c4b20)
jas_free(0x59c4b80)
jas_free(0x59c4a70)
cannot load image
jas_free(0x59c4040)
jas_free(0x59c4090)
jas_free(0x59c40e0)
jas_free(0x59c4140)
jas_free(0x59c4190)
jas_free(0x59c41e0)
jas_free(0x59c4240)
jas_free(0x59c4290)
jas_free(0x59c42e0)
jas_free(0x59c4340)
jas_free(0x59c4390)
jas_free(0x59c43e0)
jas_free(0x59c4440)
jas_free(0x59c4490)
jas_free(0x59c44e0)
jas_free(0x59c4540)
jas_free(0x59c4590)
jas_free(0x59c45e0)
jas_free(0x59c4640)
jas_free(0x59c4690)
jas_free(0x59c46e0)
jas_free(0x59c4760)
jas_free(0x59c47b0)
jas_free(0x59c4800)
jas_free(0x59c4870)
jas_free(0x59c48c0)
jas_free(0x59c4910)
jas_free(0x59c4970)
jas_free(0x59c49c0)
jas_free(0x59c4a10)
==2341== 
==2341== HEAP SUMMARY:
==2341==     in use at exit: 2,243,902 bytes in 11 blocks
==2341==   total heap usage: 53 allocs, 43 frees, 2,267,829 bytes allocated
==2341== 
==2341== LEAK SUMMARY:
==2341==    definitely lost: 168 bytes in 1 blocks
==2341==    indirectly lost: 2,243,734 bytes in 10 blocks
==2341==      possibly lost: 0 bytes in 0 blocks
==2341==    still reachable: 0 bytes in 0 blocks
==2341==         suppressed: 0 bytes in 0 blocks
==2341== Rerun with --leak-check=full to see details of leaked memory
==2341== 
==2341== For counts of detected and suppressed errors, rerun with: -v
==2341== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
mdadams commented 8 years ago

@hannob and @jridky, thank you for the above output with the extra debugging logs emabled. This result immediately told me what the reason for the double free was. A realloc call degenerates into a free call if the size parameter for realloc is 0. This was not accounted for in the stream code. What I cannot explain is why the code is being asked to create a buffer of size 0. In any case, I have fixed the problem in the stream code (although there might still be other problems). Since I cannot make the problem happen on my machine, I cannot test my fix. If someone who is having the above problem could test my fix, I would appreciate this. My fix is in commit 44a524e367597af58d6265ae2014468b334d0309 (which you can pickup if you just take the most recent code from the master branch).

Incidentally, it seems that the version of the JPEG library that I am using recognizes the JPEG file as corrupt and this causes the jasper JPEG decoder to return with an error before a buffer size of 0 is ever requested. Ofhter versions of the JPEG library (like the ones that people that see the above problem are using) seem to say that everything is okay with the JPEG file (when it is corrupt) leading to jasper doing weird stuff later. Although I cannot say for certain if the bug is in the JPEG library, I can say that its behavior seems to be different across different versions.

asarubbo commented 8 years ago

The problem seems fixed to me. At least the provided testcase does not crash anymore.

mdadams commented 8 years ago

@asarubbo: Thanks for letting me know. I am marking this bug report and the duplicate #31 as fixed then, since the data files for both bug reports now work on my machine (with the patch from my commit) and you have confirmed that the bug is fixed on your machine as well, (As it turns out, I didn't need your chroot environment after all.)

mdadams commented 8 years ago

@asarubbo Someone just reported that the attached file (i.e., the JPEG file in 1_crash.zip) fails an anti-virus scan by Symantec. The web page from Symantec says that a heuristic is used:

https://www.symantec.com/security_response/writeup.jsp?docid=2010-022301-5155-99&tabid=2

Since no other antivirus software is reporting this file as a threat, it seems likely that Symantec is giving a false positive. To be safe though, I am wondering can you vouch for the safety of this file? Did you generate it yourself? Or did it come from some unknown source? This is an issue worth considering because your JPEG file is included as test/bad/1_crash.jpg in the JasPer software distribution.

hannob commented 8 years ago

I wouldn't be overly worried by that. If you google for that what symantec detects here is not a malware, but an exploit for a vulnerability in microsoft paint. It's not unusual that a malformed file that triggers a bug in one application also triggers a bug in another one. Does the detection have any negative impact on jasper? If so then maybe tests with malformed jpeg files need to be split out of the normal distribution. In this case however you may want to pick the sample file from my duplicate bug #31 instead which doesn't seem to trigger symantec's alarm.

asarubbo commented 8 years ago

If the attached file was a virus/trojan/etc. I certainly would not say that it is :)

Jokes aside, the file has been produced by american fuzzy lop.

mdadams commented 7 years ago

@asarubbo Well, given that you are spending a lot of your time testing and reporting bugs in JasPer in order to improve the quality of the software, it would seem unlikely that you would submit a virus/trojan to distribute with JasPer. I suspected that the file was generated by a fuzzer.

@hannob In light of the fact that the file in question came from a fuzzer (and was not constructed by a malicious hadker), my only concern is the optics. Even if the file is harmless, if users think that it is not, this could negatively impact JasPer. For now, I think that I will leave this test case in the JasPer distribution. If users start showing concern about this file (in spite of it being harmless), I will simply drop it from the software distribution (and keep it only in the repository).