Global-buffer-overflow in intpl_chroma_block_ver_sse128() --> xavs2/source/common/vec/intrinsic_inter_pred.c:1592

[arayzw]

Describe the bug

Bug Relevant code as follows:

==================================================================================== void intpl_chroma_block_ver_sse128(pel_t dst, int i_dst, pel_t src, int i_src, int width, int height, const int8_t *coeff) {

int row, col;
const short offset = 32;
const int shift = 6;
int bsym = (coeff[1] == coeff[2]);
__m128i mAddOffset = _mm_set1_epi16(offset);
pel_t const *p;
__m128i mask = _mm_loadu_si128((__m128i*)(intrinsic_mask[(width & 7) - 1]));        <------ read overflow here

...... }

====================================================================================

This is a security issue.

To Reproduce

cd /path/to/xavs2/build/linux/ ./configure --enable-pic --enable-debug vim config.mak (add -fsanitize=address to CFLAGS, and -fsanitize=address -lasan to LDFLAGS) make ./xavs2 -p InputFile=./poc.yuv

ASAN Crash log

================================================================= ==103739==ERROR: AddressSanitizer: global-buffer-overflow on address 0x55c3f2947f10 at pc 0x55c3f271aae7 bp 0x7ff9a0bdd110 sp 0x7ff9a0bdd100 READ of size 16 at 0x55c3f2947f10 thread T9

0 0x55c3f271aae6 in _mm_loadu_si128 /usr/lib/gcc/x86_64-linux-gnu/9/include/emmintrin.h:703

#1 0x55c3f271aae6 in xavs2_intpl_chroma_block_ver_sse128 /home/arayz/arayz/work/xavs2/source/common/vec/intrinsic_inter_pred.c:1592
#2 0x55c3f2874214 in xavs2_intpl_chroma_block_ver_avx2 /home/arayz/arayz/work/xavs2/source/common/vec/intrinsic_inter_pred_avx2.c:1710
#3 0x55c3f2678e8f in xavs2_mc_chroma /home/arayz/arayz/work/xavs2/source/common/mc.c:823
#4 0x55c3f2613ae1 in rdo_get_pred_inter /home/arayz/arayz/work/xavs2/source/encoder/rdo.c:1654
#5 0x55c3f2613ae1 in cu_rdcost_inter /home/arayz/arayz/work/xavs2/source/encoder/rdo.c:1723
#6 0x55c3f262d105 in cu_check_skip_direct_rough2 /home/arayz/arayz/work/xavs2/source/encoder/rdo.c:2568
#7 0x55c3f262d105 in compress_cu_inter /home/arayz/arayz/work/xavs2/source/encoder/rdo.c:3068
#8 0x55c3f262d105 in xavs2_compress_ctu_inter /home/arayz/arayz/work/xavs2/source/encoder/rdo.c:3462
#9 0x55c3f26387cb in xavs2_compress_ctu_inter /home/arayz/arayz/work/xavs2/source/encoder/rdo.c:3515
#10 0x55c3f26387cb in xavs2_compress_ctu_inter /home/arayz/arayz/work/xavs2/source/encoder/rdo.c:3515
#11 0x55c3f264da84 in xavs2_lcu_row_write /home/arayz/arayz/work/xavs2/source/encoder/slice.c:436
#12 0x55c3f25c0c38 in proc_xavs2_threadpool_thread /home/arayz/arayz/work/xavs2/source/common/threadpool.c:258
#13 0x7ff9b3692608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
#14 0x7ff9b35b1132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)

0x55c3f2947f10 is located 16 bytes to the left of global variable 'intrinsic_mask' defined in '/home/arayz/arayz/work/xavs2/source/common/vec/intrinsic.c:46:1' (0x55c3f2947f20) of size 240 0x55c3f2947f10 is located 16 bytes to the right of global variable 'intrinsic_mask_256_8bit' defined in '/home/arayz/arayz/work/xavs2/source/common/vec/intrinsic.c:65:1' (0x55c3f2947d00) of size 512 SUMMARY: AddressSanitizer: global-buffer-overflow /usr/lib/gcc/x86_64-linux-gnu/9/include/emmintrin.h:703 in _mm_loadu_si128 Shadow bytes around the buggy address: 0x0ab8fe520f90: 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9 f9 0x0ab8fe520fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0ab8fe520fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0ab8fe520fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0ab8fe520fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0x0ab8fe520fe0: f9 f9[f9]f9 00 00 00 00 00 00 00 00 00 00 00 00 0x0ab8fe520ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0ab8fe521000: 00 00 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 00 00 0x0ab8fe521010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0ab8fe521020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0ab8fe521030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc Thread T9 created by T0 here:

0 0x7ff9b3836815 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208

#1 0x55c3f25b8df2 in xavs2_create_thread /home/arayz/arayz/work/xavs2/source/common/common.c:323

==103739==ABORTING

Additional context

• OS: Ubuntu 20.04 (Desktop)
• Compiler: gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)

PoC: poc.zip