Large allocation by yr_compiler_load_atom_quality_table

[3-24]

Describe the bug We found that yara is trying to allocate 0x7ffff...f bytes by running some fuzzer.

To Reproduce We built yara with address sanitizer:

CFLAGS="-fsanitize=address -g -O0" CXXFLAGS="-fsanitize=address -g -O0" ./configure

The command input is,

yara "" "--atom-quality-table=." ""

Stack Trace

==27573==WARNING: AddressSanitizer failed to allocate 0x7fffffffffffffff bytes
==27573==AddressSanitizer's allocator is terminating the process instead of returning 0
==27573==If you don't like this behavior set allocator_may_return_null=1
==27573==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_allocator.cc:218 "((0)) != (0)" (0x0, 0x0)
    #0 0x7ffff6f01bf2  (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe9bf2)
    #1 0x7ffff6f20575 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x108575)
    #2 0x7ffff6f07332  (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xef332)
    #3 0x7ffff6e3fe46  (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x27e46)
    #4 0x7ffff6ef6b0a in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xdeb0a)
    #5 0x7ffff6d5f7dd in yr_malloc libyara/mem.c:132
    #6 0x7ffff6d1d531 in yr_compiler_load_atom_quality_table libyara/compiler.c:428
    #7 0x555555562056 in main cli/yara.c:1519
    #8 0x7ffff663dc86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86)
    #9 0x55555555b6d9 in _start (/home/youngseok/latest-subjects/yara/.libs/lt-yara+0x76d9)

Environment:

• OS: Ubuntu 18.04
• gcc: 7.5.0
• YARA version: 4.3.0 (master branch, commit id 313ab8080ad26efdf626d84c3b7b62c151ff295f)