Buffer-overflow in fy_utf8_get_right() when calling fy_document_insert_at()

[bladchan]

Hi,

I am running some experiments for AFLAPI(fuzzing) and it has found a buffer-overflow (to be exact, out-of-bounds access after debugging) in fy_atom_raw_line_iter_next. This bug seems to be harmless because it happened when insert with an invalid character in alias (fy_diag_error_atom_display-->fy_atom_raw_line_iter_next --> fy_utf8_get_right).

Environment: Ubuntu 20.04 + gcc 9.4.0

I have debugged it a few hours ago, but cannot find what really cause this bug. But I found that this bug access out-of-bounds in fy-utf8.h/line 93, so this bug seems to be harmless.

Harness (attached: file named as "test_fy_document_insert_at.c"):

#include <libfyaml.h>
#include <stdio.h>

int main(int argc, char** argv) {

    struct fy_document *fyd = NULL;
    fyd = fy_document_build_from_file(NULL, "test1.yaml");
    if (!fyd) {
        fprintf(stderr, "failed to build document");
        goto failed;
    }
    int rc;

    char key[12] = {0x26, 0x2b, 0x74, 0x68, 0x65, 0x62, 0x65, 0x86, 0x6e, 0x67, 0x77, 0x00}; // here is the poc (len: 0xc, but access position 0xd?)

    rc = fy_document_insert_at(fyd, key, FY_NT, fy_node_buildf(fyd, "abc"));

    if (rc) {
        fprintf(stderr, "failed to emit document to stdout\n");
        goto failed;
    }

    rc = fy_emit_document_to_fp(fyd, FYECF_DEFAULT | FYECF_SORT_KEYS, stdout);
    if (rc) {
        fprintf(stderr, "failed to emit document to stdout\n");
        goto failed;
    }

failed:
    fy_document_destroy(fyd);
    return rc;

}

The test1.yaml (attached):

base: &base
    name: this-is-a-name

Poc: Poc.zip

To reproduce: • Complie the hole project with ASAN:

CFLAGS="-fsanitize=address -g" ./bootstrap.sh
CFLAGS="-fsanitize=address -g" ./configure
make && sudo make install

• Complie the harness with ASAN:

gcc -fsanitize=address -o test_fy_document_insert_at test_fy_document_insert_at.c -lfyaml

• Run harness:

./test_fy_document_insert_at

ASAN says:

<memory-@0x7ffd52519e40-0x7ffd52519e4a>:1:14: error: invalid character in anchor
=================================================================
==1614159==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffd52519e4c at pc 0x7f0b71a150ce bp 0x7ffd52517ba0 sp 0x7ffd52517b90
READ of size 1 at 0x7ffd52519e4c thread T0
    #0 0x7f0b71a150cd in fy_utf8_get_right lib/fy-utf8.h:93
    #1 0x7f0b71a150cd in fy_atom_raw_line_iter_next lib/fy-atom.c:1732
    #2 0x7f0b71a04339 in fy_diag_error_atom_display lib/fy-diag.c:661
    #3 0x7f0b71a06236 in fy_diag_error_token_display lib/fy-diag.c:806
    #4 0x7f0b71a06236 in fy_diag_error_token_display lib/fy-diag.c:801
    #5 0x7f0b71a06236 in fy_diag_vreport lib/fy-diag.c:854
    #6 0x7f0b71a06eee in fy_parser_diag_vreport lib/fy-diag.c:963
    #7 0x7f0b71a0706f in fy_parser_diag_report lib/fy-diag.c:976
    #8 0x7f0b719d7a6d in fy_fetch_anchor_or_alias lib/fy-parse.c:2894
    #9 0x7f0b719f1c77 in fy_fetch_tokens lib/fy-parse.c:4976
    #10 0x7f0b719f4057 in fy_scan_peek lib/fy-parse.c:5093
    #11 0x7f0b719f4057 in fy_scan_peek lib/fy-parse.c:5038
    #12 0x7f0b719f7144 in fy_parse_internal lib/fy-parse.c:5524
    #13 0x7f0b71a4ebff in fy_document_builder_load_document lib/fy-docbuilder.c:529
    #14 0x7f0b71a456be in fy_parse_load_document_with_builder lib/fy-doc.c:1940
    #15 0x7f0b71a45acd in fy_document_build_internal lib/fy-doc.c:3242
    #16 0x7f0b71a45e9f in fy_document_build_from_string lib/fy-doc.c:3299
    #17 0x7f0b71a4603b in fy_node_mapping_lookup_pair_by_string lib/fy-doc.c:3793
    #18 0x7f0b71a4603b in fy_node_mapping_lookup_pair_by_string lib/fy-doc.c:3784
    #19 0x7f0b71a4609c in fy_node_mapping_lookup_by_string lib/fy-doc.c:3810
    #20 0x7f0b71a434a4 in fy_node_by_path_internal lib/fy-doc.c:4202
    #21 0x7f0b71a47171 in fy_node_by_path lib/fy-doc.c:4467
    #22 0x7f0b71a47386 in fy_document_insert_at lib/fy-doc.c:2484
    #23 0x55bc66b654a7 in main (/home/ubuntu/test/libfyaml/fuzz/test_fy_document_insert_at+0x14a7)
    #24 0x7f0b717c8082 in __libc_start_main ../csu/libc-start.c:308
    #25 0x55bc66b6522d in _start (/home/ubuntu/test/libfyaml/fuzz/test_fy_document_insert_at+0x122d)

Address 0x7ffd52519e4c is located in stack of thread T0 at offset 44 in frame
    #0 0x55bc66b652f8 in main (/home/ubuntu/test/libfyaml/fuzz/test_fy_document_insert_at+0x12f8)

  This frame has 1 object(s):
    [32, 44) 'key' (line 14) <== Memory access at offset 44 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow lib/fy-utf8.h:93 in fy_utf8_get_right
Shadow bytes around the buggy address:
  0x10002a49b370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10002a49b380: 00 00 00 00 00 00 f1 f1 f1 f1 f1 f1 04 f2 00 f2
  0x10002a49b390: f2 f2 04 f3 f3 f3 00 00 00 00 00 00 00 00 00 00
  0x10002a49b3a0: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 f3 f3 f3
  0x10002a49b3b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10002a49b3c0: 00 00 00 00 f1 f1 f1 f1 00[04]f3 f3 00 00 00 00
  0x10002a49b3d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10002a49b3e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10002a49b3f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10002a49b400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10002a49b410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==1614159==ABORTING

[bladchan]

When calling fy_document_build_from_file, there is also an unknown-crash in fy_utf8_get_right().

Harness (attached: file named as "test_fy_document_build_from_file.c"):

#include <libfyaml.h>
#include <stdio.h>

int main(int argc, char** argv) {

    if(argc != 2) return 0;

    struct fy_document *fyd = NULL;
    fyd = fy_document_build_from_file(NULL, argv[1]);
    if (!fyd) {
        fprintf(stderr, "failed to build document");
        goto failed;
    }

failed:
    fy_document_destroy(fyd);
    return 0;

}

Poc: Poc1.zip

To reproduce: The same way above.

• Run harness:

./test_fy_document_build_from_file ./unknown_crash.yaml

ASAN says:

unknown_crash.yaml:1:11923: error: invalid character in anchor
=================================================================
==1614509==ERROR: AddressSanitizer: unknown-crash on address 0x7f5f4fd26e91 at pc 0x7f5f52cb10ce bp 0x7fff6c1b7580 sp 0x7fff6c1b7570
READ of size 1 at 0x7f5f4fd26e91 thread T0
    #0 0x7f5f52cb10cd in fy_utf8_get_right lib/fy-utf8.h:93
    #1 0x7f5f52cb10cd in fy_atom_raw_line_iter_next lib/fy-atom.c:1732
    #2 0x7f5f52ca0339 in fy_diag_error_atom_display lib/fy-diag.c:661
    #3 0x7f5f52ca2236 in fy_diag_error_token_display lib/fy-diag.c:806
    #4 0x7f5f52ca2236 in fy_diag_error_token_display lib/fy-diag.c:801
    #5 0x7f5f52ca2236 in fy_diag_vreport lib/fy-diag.c:854
    #6 0x7f5f52ca2eee in fy_parser_diag_vreport lib/fy-diag.c:963
    #7 0x7f5f52ca306f in fy_parser_diag_report lib/fy-diag.c:976
    #8 0x7f5f52c73a6d in fy_fetch_anchor_or_alias lib/fy-parse.c:2894
    #9 0x7f5f52c8dc77 in fy_fetch_tokens lib/fy-parse.c:4976
    #10 0x7f5f52c90057 in fy_scan_peek lib/fy-parse.c:5093
    #11 0x7f5f52c90057 in fy_scan_peek lib/fy-parse.c:5038
    #12 0x7f5f52c9528a in fy_parse_internal lib/fy-parse.c:6034
    #13 0x7f5f52ceabff in fy_document_builder_load_document lib/fy-docbuilder.c:529
    #14 0x7f5f52ce16be in fy_parse_load_document_with_builder lib/fy-doc.c:1940
    #15 0x7f5f52ce1acd in fy_document_build_internal lib/fy-doc.c:3242
    #16 0x7f5f52ce230c in fy_document_build_from_file lib/fy-doc.c:3320
    #17 0x555ceb72828b in main (/home/ubuntu/test/libfyaml/fuzz/test_fy_document_build_from_file+0x128b)
    #18 0x7f5f52a64082 in __libc_start_main ../csu/libc-start.c:308
    #19 0x555ceb72816d in _start (/home/ubuntu/test/libfyaml/fuzz/test_fy_document_build_from_file+0x116d)

Address 0x7f5f4fd26e91 is a wild pointer.
SUMMARY: AddressSanitizer: unknown-crash lib/fy-utf8.h:93 in fy_utf8_get_right
Shadow bytes around the buggy address:
  0x0fec69f9cd80: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9cd90: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9cda0: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9cdb0: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9cdc0: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
=>0x0fec69f9cdd0: fe fe[fe]fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9cde0: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9cdf0: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9ce00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9ce10: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
  0x0fec69f9ce20: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==1614509==ABORTING

I think this two bugs caused by the same reason, so I report this crash here. Hope that helps. 😄

[pantoniou]

Again, fixed after a398979df30fe65abd69400b6170b938c6709004

Please report any additional failures you can find.

I will let this issue open until I hear back from you and confirm that it was fixed.

[bladchan]

Thanks for replying.

After testing with all buffer-overflow testcases, ASAN cannot detect buffer-overflow issues.

The patch https://github.com/pantoniou/libfyaml/commit/a398979df30fe65abd69400b6170b938c6709004 fixed the buffer-overflow issues mentioned above.