Blocked branch hyperlink in OpenSSL report

I found a problem related to the hyperlink of blocked branch in OpenSSL report. This could lead to two pieces of mis-leading information, wrong link to the coverage report and false positives in fuzz blockers identification. Also, the current openssl report is missing, I am taking reference from the past report.

Three OpenSSL versions are being fuzzed, they should have been completely independent from each other. The naming pattern for fuzzers should be that FUZZ_111, FUZZ_30 and FUZZ working on OpenSSL 1.1.1, OpenSSL 3.0 and the latest version, respectively. However, if we click into some of the detailed report, the blocked branch is linking to the wrong version, instead of the version being tested. For example looking at a fuzzer on OpenSSL 3.0 (https://storage.googleapis.com/oss-fuzz-introspector/openssl/inspector-report/20230201/calltree_view_13.html). You can see blocked branch link starting with "/src/openssl111/", links like this won't direct us to the code coverage page, the correct link should start with "/src/openssl30/*". Manually changing version numbers gets you to the right page.

Besides, some fuzz blockers might be false positives. Please have a look at the first fuzz blocker of the bignum fuzzer (https://storage.googleapis.com/oss-fuzz-introspector/openssl/inspector-report/20230201/fuzz_report.html#fuzz_blocker16) working on OpenSSL_latest. The blocked branch at https://storage.googleapis.com/oss-fuzz-coverage/openssl/reports-by-target/20230201/bignum/linux/src/openssl/crypto/bn/bn_ctx.c.html#L122 has one line of unreached code, return NULL;. P.S. you need to manually change the link to access this one. However, the report identified unreached functions and reported this as a fuzz blocker.

The same code location in OpenSSL 3.0 has one more line of unreached error handling function so that it is identified as a fuzz blocker in fuzzer bignum_30. Please see the fuzz blocker report at https://storage.googleapis.com/oss-fuzz-introspector/openssl/inspector-report/20230201/fuzz_report.html#fuzz_blocker13, and the coverage report at https://storage.googleapis.com/oss-fuzz-coverage/openssl/reports-by-target/20230201/bignum_30/linux/src/openssl30/crypto/bn/bn_ctx.c.html#L122.

In my opinion, not only the link has the wrong version numbers, the identification of fuzz blockers might be looking at coverage reports in the wrong version as well. Therefore, the first blocker of fuzzer bignum on OpenSSL_latest is still reported, despite the fact that only return NULL is blocked.

Could you please have a look at this. Many thanks.

Thanks a lot for the thorough report @wenta0g !!

1) Ack on the folder issue -- I'll see what goes on there as it's clearly a bug. 2) About the potential false positive of the branch blockers. I think by nature the branch blockers can have a form of "indirectness" in them, which means you see issues as the one you point out where seemingly only a single return NULL line is blocked but this is determined to have some wider effect. I'll see if I can dig more into why this specific branch blocker is labelled a branch blocker.

In this context, you might be interested in some recent improvements that make it possible to programmatically query OSS-Fuzz Fuzz Introspector reports, and also make's it easy to develop your own tools that query/inspect fuzz introspector data by OSS-Fuzz:

The following folder https://github.com/ossf/fuzz-introspector/tree/main/tools/oss-fuzz-scanner holds a small utility scanner.py which can be used to: (1) find oss-fuzz fuzz introspector raw data from the builds, this is the fuzz introspector data created during project compilation/building, i.e. form the frontend/ directories; (2) download the raw data; (3) run fuzz introspector locally on the raw data; (4) inspect the fuzz introspector reports locally using your own tooling.

For example, the following short script will download the latest fuzz introspector raw data from a specific project, go through each of the profiles of the project and print out how many branch blockers there are for the given fuzzer:

import sys
import scanner

def print_function_details(project_name):
    # Get an iterator for all reports in the last 100 days
    report_generator = scanner.get_all_reports([project_name],
                                               days_to_analyse=100,
                                               interval_size=1)

    # Only use the first working report
    project, date_as_str, introspector_project = next(report_generator)
    for profile in introspector_project.profiles:
        print("Profile: %s has %d blokers"%(profile.identifier, len(profile.branch_blockers)))

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print(
            "Usage: python3 branch_blocker_inspector.py project_name")
        sys.exit(0)
    project_name = sys.argv[1]
    print_function_details(project_name)

if you place the above code in the above directory in a file called branch_blocker_inspector.py then you run queries like:

$ python3 ./branch_blocker_inspector.py htslib
Profile: /src/htslib/test/fuzz/hts_open_fuzzer.c has 664 blokers

$ python3 ./branch_blocker_inspector.py croaring
Profile: /src/croaring_fuzzer.c has 111 blokers
Profile: /src/croaring_fuzzer_cc.cc has 111 blokers

Remember to install the python dependencies in requirements.txt as your machine will be running fuzz introspector locally -- although only the post-processing bit (the code in src/) and not compiling any code or that stuff.

The introspector_project in the above code gives you access to https://github.com/ossf/fuzz-introspector/blob/caa4fe38158cb33e26d56f340bfa7951055ae516/src/fuzz_introspector/analysis.py#L39 which includes almost all analyses done by fuzz introspector. This class has particularly two elements of interest: proj_profile and profiles which give access to the project overall and the individual fuzzers in the project, respectively.

Then, you can start to create more queries into the branch blockers, for example instead of just printing the amount of branch blockers you can start to inspect what code it is estimated to block (https://github.com/ossf/fuzz-introspector/blob/main/src/fuzz_introspector/datatypes/branch_profile.py) as a way of understanding it deeper.

ossf / fuzz-introspector

Blocked branch hyperlink in OpenSSL report #967