Global-buffer-overflow in cu_get_quant_params() --> davs2/source/common/quant.h:115

[arayzw]

Describe the bug

=============================================================================================

static ALWAYS_INLINE void cu_get_quant_params(davs2_t h, int qp, int bit_size, int shift, int scale) { shift = IQ_SHIFT[qp] + (h->sample_bit_depth + 1) + bit_size - LIMIT_BIT; <------ out of bounds here *scale = IQ_TAB[qp]; }

=============================================================================================

To Reproduce

cd /path/to/davs2/build/linux/ ./configure --enable-pic vim config.mak (add -fsanitize=address to CFLAGS, and -fsanitize=address -lasan to LDFLAGS) make ./davs2 -i /path/to/poc4 -o test.yuv

ASAN Crash log

================================================================= ==107413==ERROR: AddressSanitizer: global-buffer-overflow on address 0x55ec49b52e7c at pc 0x55ec49a5bb2d bp 0x7fabbb269cb0 sp 0x7fabbb269ca0 READ of size 2 at 0x55ec49b52e7c thread T1

0 0x55ec49a5bb2c in cu_get_quant_params /home/arayz/arayz/work/davs2/source/common/quant.h:115

#1 0x55ec49a5bb2c in cu_read_all_coeffs /home/arayz/arayz/work/davs2/source/common/cu.cc:662
#2 0x55ec49a5bb2c in cu_read_info /home/arayz/arayz/work/davs2/source/common/cu.cc:872
#3 0x55ec49a5bb2c in davs2_decode_lcu_parse /home/arayz/arayz/work/davs2/source/common/cu.cc:1327
#4 0x55ec49a55deb in davs2_decode_lcu_parse /home/arayz/arayz/work/davs2/source/common/cu.cc:1316
#5 0x55ec49a55deb in davs2_decode_lcu_parse /home/arayz/arayz/work/davs2/source/common/cu.cc:1316
#6 0x55ec49a55deb in davs2_decode_lcu_parse /home/arayz/arayz/work/davs2/source/common/cu.cc:1316
#7 0x55ec499abb6b in decode_one_lcu_row /home/arayz/arayz/work/davs2/source/common/decoder.cc:438
#8 0x55ec499b12ee in davs2_decoder_decode_picture_data /home/arayz/arayz/work/davs2/source/common/decoder.cc:1168
#9 0x55ec499d8b6a in davs2_threadpool_thread /home/arayz/arayz/work/davs2/source/common/threadpool.cc:216
#10 0x7fabce4aa608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
#11 0x7fabce083132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)

0x55ec49b52e7c is located 4 bytes to the left of global variable 'dmh_pos' defined in '/home/arayz/arayz/work/davs2/source/common/cu.cc:77:21' (0x55ec49b52e80) of size 36 0x55ec49b52e7c is located 28 bytes to the right of global variable 'davs2_IQ_SHIFT' defined in '/home/arayz/arayz/work/davs2/source/common/cu.cc:91:15' (0x55ec49b52dc0) of size 160 SUMMARY: AddressSanitizer: global-buffer-overflow /home/arayz/arayz/work/davs2/source/common/quant.h:115 in cu_get_quant_params Shadow bytes around the buggy address: 0x0abe09362570: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00 0x0abe09362580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 0x0abe09362590: f9 f9 f9 f9 00 00 00 00 00 00 f9 f9 f9 f9 f9 f9 0x0abe093625a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0abe093625b0: 00 00 00 00 f9 f9 f9 f9 00 00 00 00 00 00 00 00 =>0x0abe093625c0: 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9[f9] 0x0abe093625d0: 00 00 00 00 04 f9 f9 f9 f9 f9 f9 f9 00 00 00 00 0x0abe093625e0: 00 00 00 00 f9 f9 f9 f9 00 00 00 00 00 00 00 00 0x0abe093625f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0abe09362600: 00 00 00 00 00 00 00 00 f9 f9 f9 f9 00 00 00 00 0x0abe09362610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc Thread T1 created by T0 here:

0 0x7fabce4ff815 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208

#1 0x55ec499d8f36 in davs2_threadpool_init /home/arayz/arayz/work/davs2/source/common/threadpool.cc:271

==107413==ABORTING

Additional context

• OS: Ubuntu 20.04 (Desktop)
• Compiler: gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)

PoC: poc4.zip