Using ASAN, a heap-buffer-overflow is reported when using stbir_resize with STBIR_2CHANNEL. The issue only happens when using specific source-destination image dimensions.
The issue only happens when compiling with -mavx2. ASAN message shows _mm_loadu_ps trying to load past the heap red leftzone. Apparently decode_buffer_size in stbir__alloc_internal_mem_and_build_samplers is not being correctly calculated for this particular case.
I've attached a minimum reproducible example that triggers the issue.
example.zip
==161245==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x522000004470 at pc 0x5a1bedc22660 bp 0x7ffe1dbc5590 sp 0x7ffe1dbc5580
READ of size 16 at 0x522000004470 thread T0
#0 0x5a1bedc2265f in _mm_loadu_ps /usr/lib/gcc/x86_64-pc-linux-gnu/14.2.1/include/xmmintrin.h:940
#1 0x5a1bedc2265f in stbir__horizontal_gather_2_channels_with_9_coeffs /home/ien/Projects/stbir_repro/stb_image_resize2.h:10341
#2 0x5a1bedc89893 in stbir__resample_horizontal_gather /home/ien/Projects/stbir_repro/stb_image_resize2.h:6158
#3 0x5a1bedc89e25 in stbir__resample_vertical_gather /home/ien/Projects/stbir_repro/stb_image_resize2.h:6194
#4 0x5a1bedc8a8d5 in stbir__vertical_gather_loop /home/ien/Projects/stbir_repro/stb_image_resize2.h:6276
#5 0x5a1bedc91c85 in stbir__perform_resize /home/ien/Projects/stbir_repro/stb_image_resize2.h:7262
#6 0x5a1bedc96674 in stbir_resize_extended /home/ien/Projects/stbir_repro/stb_image_resize2.h:7886
#7 0x5a1bedc978ea in stbir_resize /home/ien/Projects/stbir_repro/stb_image_resize2.h:8056
#8 0x5a1bedc97bdc in main /home/ien/Projects/stbir_repro/main.c:21
#9 0x78e191e34e07 (/usr/lib/libc.so.6+0x25e07) (BuildId: 98b3d8e0b8c534c769cb871c438b4f8f3a8e4bf3)
#10 0x78e191e34ecb in __libc_start_main (/usr/lib/libc.so.6+0x25ecb) (BuildId: 98b3d8e0b8c534c769cb871c438b4f8f3a8e4bf3)
#11 0x5a1bedba3394 in _start (/home/ien/Projects/stbir_repro/main+0xa394) (BuildId: 6d33a73a79f0debd118c6c9367ad17c7dd630781)
0x52200000447c is located 0 bytes after 4988-byte region [0x522000003100,0x52200000447c)
allocated by thread T0 here:
#0 0x78e1920fd891 in malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69
#1 0x5a1bedc90063 in stbir__alloc_internal_mem_and_build_samplers /home/ien/Projects/stbir_repro/stb_image_resize2.h:7074
#2 0x5a1bedc9601e in stbir__perform_build /home/ien/Projects/stbir_repro/stb_image_resize2.h:7801
#3 0x5a1bedc963e9 in stbir_build_samplers_with_splits /home/ien/Projects/stbir_repro/stb_image_resize2.h:7841
#4 0x5a1bedc9640f in stbir_build_samplers /home/ien/Projects/stbir_repro/stb_image_resize2.h:7851
#5 0x5a1bedc96567 in stbir_resize_extended /home/ien/Projects/stbir_repro/stb_image_resize2.h:7868
#6 0x5a1bedc978ea in stbir_resize /home/ien/Projects/stbir_repro/stb_image_resize2.h:8056
#7 0x5a1bedc97bdc in main /home/ien/Projects/stbir_repro/main.c:21
#8 0x78e191e34e07 (/usr/lib/libc.so.6+0x25e07) (BuildId: 98b3d8e0b8c534c769cb871c438b4f8f3a8e4bf3)
#9 0x78e191e34ecb in __libc_start_main (/usr/lib/libc.so.6+0x25ecb) (BuildId: 98b3d8e0b8c534c769cb871c438b4f8f3a8e4bf3)
#10 0x5a1bedba3394 in _start (/home/ien/Projects/stbir_repro/main+0xa394) (BuildId: 6d33a73a79f0debd118c6c9367ad17c7dd630781)
SUMMARY: AddressSanitizer: heap-buffer-overflow /usr/lib/gcc/x86_64-pc-linux-gnu/14.2.1/include/xmmintrin.h:940 in _mm_loadu_ps
Shadow bytes around the buggy address:
0x522000004180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x522000004200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x522000004280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x522000004300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x522000004380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x522000004400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00[00]04
0x522000004480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x522000004500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x522000004580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x522000004600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x522000004680: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==161245==ABORTING
Using ASAN, a heap-buffer-overflow is reported when using
stbir_resize
with STBIR_2CHANNEL. The issue only happens when using specific source-destination image dimensions.The issue only happens when compiling with
-mavx2
. ASAN message shows_mm_loadu_ps
trying to load past the heap red leftzone. Apparentlydecode_buffer_size
instbir__alloc_internal_mem_and_build_samplers
is not being correctly calculated for this particular case.I've attached a minimum reproducible example that triggers the issue. example.zip