google / atheris

Apache License 2.0
1.35k stars 112 forks source link

Coverage doesn't increase #23

Closed jvoisin closed 2 years ago

jvoisin commented 2 years ago

I'm running the following fuzzer for mat2 in a virtualenv:

import os
import sys

import atheris

with atheris.instrument_imports():
    from libmat2 import parser_factory

def TestOneInput(data):
    with open('/tmp/mat2_fuzz', 'wb') as f:
        f.write(data)
    try:
        p, _ = parser_factory.get_parser('/tmp/mat2_fuzz')
        if p:
            p.get_meta()
            p.remove_all()
            p, _ = parser_factory.get_parser('/tmp/mat2_fuzz')
            p.get_meta()
    except ValueError:
        pass
    os.remove('/tmp/mat2_fuzz')

atheris.Setup(sys.argv, TestOneInput)
atheris.Fuzz()

and got the following results:

(ven) jvoisin@grimhilde 18:08 ~/dev/mat2 python3 fuzz.py ./tests/data/
INFO: Instrumenting libmat2
INFO: Instrumenting libmat2.exiftool
INFO: Instrumenting json
INFO: Instrumenting json.decoder
INFO: Instrumenting json.scanner
INFO: Instrumenting json.encoder
INFO: Instrumenting logging
INFO: Instrumenting traceback
INFO: Instrumenting linecache
INFO: Instrumenting tokenize
INFO: Instrumenting token
INFO: Instrumenting weakref
INFO: Instrumenting _weakrefset
INFO: Instrumenting string
INFO: Instrumenting _string
WARNING: It looks like this module is imported by a custom loader. Atheris has experimental support for this. However, it may be incompatible with certain libraries. If you experience unusual errors or poor coverage collection, try atheris.instrument_all() instead, add enable_loader_override=False to instrument_imports(), or file an issue on GitHub.
INFO: Instrumenting threading
INFO: Instrumenting atexit
INFO: Instrumenting shutil
INFO: Instrumenting fnmatch
INFO: Instrumenting errno
INFO: Instrumenting zlib
INFO: Instrumenting bz2
INFO: Instrumenting _compression
INFO: Instrumenting lzma
INFO: Instrumenting pwd
INFO: Instrumenting grp
INFO: Instrumenting subprocess
INFO: Instrumenting signal
INFO: Instrumenting _posixsubprocess
INFO: Instrumenting select
INFO: Instrumenting selectors
INFO: Instrumenting math
INFO: Instrumenting libmat2.abstract
INFO: Instrumenting libmat2.bubblewrap
INFO: Instrumenting tempfile
INFO: Instrumenting random
INFO: Instrumenting bisect
INFO: Instrumenting _bisect
INFO: Instrumenting _random
INFO: Instrumenting _sha512
INFO: Instrumenting libmat2.video
INFO: Instrumenting libmat2.parser_factory
INFO: Instrumenting glob
INFO: Instrumenting mimetypes
INFO: Instrumenting urllib
INFO: Instrumenting urllib.parse
INFO: Instrumenting libmat2.images
INFO: Instrumenting imghdr
INFO: Instrumenting cairo
INFO: Instrumenting gi
INFO: Instrumenting pkgutil
INFO: Instrumenting gi._error
INFO: Instrumenting gi.repository
INFO: Instrumenting gi.importer
INFO: Instrumenting gi.module
INFO: Instrumenting gi.types
INFO: Instrumenting gi._constants
INFO: Instrumenting gi.docstring
INFO: Instrumenting gi._propertyhelper
INFO: Instrumenting gi._signalhelper
INFO: Instrumenting gi.overrides
INFO: Instrumenting gi.overrides.GLib
INFO: Instrumenting gi.overrides.GLib
INFO: Instrumenting socket
INFO: Instrumenting _socket
INFO: Instrumenting array
INFO: Instrumenting gi._ossighelper
INFO: Instrumenting __future__
INFO: Instrumenting gi._option
INFO: Instrumenting optparse
INFO: Instrumenting textwrap
INFO: Instrumenting gettext
INFO: Instrumenting locale
INFO: Instrumenting gi.overrides.GObject
INFO: Instrumenting gi.overrides.GObject
INFO: Instrumenting gi.overrides.Gio
INFO: Instrumenting gi.overrides.Gio
INFO: Instrumenting gi.overrides.GdkPixbuf
INFO: Instrumenting gi.overrides.GdkPixbuf
INFO: Instrumenting libmat2.epub
INFO: Instrumenting uuid
INFO: Instrumenting platform
INFO: Instrumenting zipfile
INFO: Instrumenting binascii
INFO: Instrumenting struct
INFO: Instrumenting _struct
INFO: Instrumenting xml
INFO: Instrumenting xml.etree
INFO: Instrumenting xml.etree.ElementTree
INFO: Instrumenting xml.etree.ElementPath
INFO: Instrumenting _elementtree
INFO: Instrumenting copy
INFO: Instrumenting pyexpat
INFO: Instrumenting libmat2.archive
INFO: Instrumenting datetime
INFO: Instrumenting _datetime
INFO: Instrumenting tarfile
INFO: Instrumenting libmat2.office
INFO: Instrumenting libmat2.torrent
INFO: Instrumenting libmat2.harmless
INFO: Instrumenting libmat2.audio
INFO: Instrumenting mutagen
INFO: Instrumenting mutagen._util
INFO: Instrumenting decimal
INFO: Instrumenting numbers
INFO: Instrumenting mutagen._file
INFO: Instrumenting mutagen._tags
INFO: Instrumenting libmat2.pdf
INFO: Instrumenting distutils
INFO: Instrumenting distutils.version
INFO: Instrumenting libmat2.web
INFO: Instrumenting html
INFO: Instrumenting html.entities
INFO: Instrumenting html.parser
INFO: Instrumenting _markupbase
INFO: Using built-in libfuzzer
WARNING: Failed to find function "__sanitizer_acquire_crash_state".
WARNING: Failed to find function "__sanitizer_print_stack_trace".
WARNING: Failed to find function "__sanitizer_set_death_callback".
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3911405642
INFO: Loaded 1 modules   (14598 inline 8-bit counters): 14598 [0x10d4970, 0x10d8276), 
INFO: Loaded 1 PC tables (14598 PCs): 14598 [0x10f0650,0x11296b0), 
INFO:       50 files found in ./tests/data/
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 50 min: 1b max: 4383613b total: 10698367b rss: 55Mb
#51 INITED cov: 20 ft: 20 corp: 1/1b exec/s: 0 rss: 59Mb
#32768  pulse  cov: 20 ft: 20 corp: 1/1b lim: 325 exec/s: 10922 rss: 59Mb
#65536  pulse  cov: 20 ft: 20 corp: 1/1b lim: 652 exec/s: 9362 rss: 59Mb
#131072 pulse  cov: 20 ft: 20 corp: 1/1b lim: 1300 exec/s: 9362 rss: 59Mb
#262144 pulse  cov: 20 ft: 20 corp: 1/1b lim: 2611 exec/s: 9362 rss: 59Mb
#524288 pulse  cov: 20 ft: 20 corp: 1/1b lim: 5212 exec/s: 9362 rss: 59Mb
…

I tried with enable_loader_override=False, but it didn't change anything.

am I doing something wrong?

jvoisin commented 2 years ago

Problem solved, mat2 uses file extensions to guess file format, sorry for the noise.

jvoisin commented 2 years ago

And it found some crashes, nice!