Setting up a Fuzz infrastructure for a cyber-secure software development environment

[ThibFrgsGmz]

F´ Version	devel
Affected Component	Testing, Code Coverage, Cyber security

---

Feature Description

F´ shall have its own fuzzing infrastructure capable of running the Fuzz test harness continuously.

Fuzzing is used to complement the explicit testing of functional requirements (unit and integration tests) of the mission.

As a first step, the idea is to invite project stakeholders to think about this fuzzing infrastructure and see what can be implemented in F´.

This would require:

1. Choose the Fuzz Engine to use (see here), such as :

   • AFLPlusPlus, an improvement of the Google's AFL used on cFS.
   • LLVM's LibFuzzer notably implemented in Jason Turner's cpp_starter_project project.

2. Develop a process/mechanism for Fuzzing. Should we use fprime-util like for unit testing? [cf here]

3. Develop a standard Fuzz test harness template/pattern to apply to all components.

4. Implement a Fuzz test harness on an interesting but simple component of F´.

5. Specify the duration, frequency of this infrastructure activation (24 hours a day, 7 days a week? every night?).

6. Configure a continuous integration pipeline to run the Fuzz test harness.

7. See how the Fuzzing results are saved/displayed to the community.

8. Create a SECURITY.md file containing evidence that F´ has a cyber-secure software development environment.

Rationale

Fuzz testing is considered an important tool in the software development process to protect assets in security related and safety critical applications.

By combining the fuzzing infrastructure with the LGTM code analysis platform that prevents critical vulnerabilities, F´ would have a cyber-secure software development environment suitable for today's concerns.

Indeed, fuzzing is an effective method to detect bugs before they are discovered in operation and exploited by malicious agents.

Fuzz tests detect programming errors that have nothing to do with the project requirements, such as [ref]:

• C/C++-specific bugs:
  • Use-after-free, buffer overflows
  • Uninitialized memory uses
  • Memory leaks
• Arithmetic bugs:
  • Divisions by zero, int/float overflows, invalid bitwise shifts.
• Pure and simple crashes:
  • NULL dereferences, undetected exceptions
• Concurrency bugs:
  • Data races, deadlocks
• Resource utilization problems:
  • Memory exhaustion, hanging or infinite loops, infinite recursion (stack overflows).
• Logic bugs:
  • Discrepancies between two implementations of the same protocol.
  • Round-trip consistency bugs
  • Assertion failures

Unit or integration test harnesses are rarely written to try to catch these types of bugs, which are primarily responsible for security flaws and reliability problems.

[ThibFrgsGmz]

Related discussion: https://github.com/nasa/fprime/discussions/1222

The description of the feature only reflects my thoughts. I'm sure it can be improved using a different approach, we can obviously discuss the development plan/roadmap.

Why not create a [Call For Comments] so that stakeholders can share their ideas more easily? This would be easier than in this issue and in a dedicated thread.

[ThibFrgsGmz]

According to the Open Source Security Foundation (OpenSSF) the implementation of Fuzzing is taken into account when measuring the security health of a project.

The risk associated with the non-presence of Fuzzing is medium (possible vulnerabilities in the code).

[Joshua-Anderson]

Fuzzing common F' commands (I.E. file uplink, command sequencer) probably wouldn't provide much security value (if you can command the FSW, you've already been compromised) but would be an excellent way to find bugs that might be missed by unit tests. Components should be designed in such a way that no ground command input should be able to crash or assert the FSW.

This could provide good returns for effort since you could setup generic command fuzzing support than turn it loose across all the common components and see what it finds.

[astroesteban]

FYI: I was looking into fuzzing and I discovered that the creator of the Google AFL library has a new version: https://github.com/AFLplusplus/AFLplusplus

The reason the author provided for creating a new AFL project is that the original one under Google doesn't get updated as frequently as it should. Looks like the last change to Google's AFL library was 2 years ago.

[ThibFrgsGmz]

I found some new information in Google's Fuschia OS documentation about integrating an existing fuzzing infrastructure.

Integrating a project into a fuzzing infrastructure would require considerable effort, but has many advantages. With a complete fuzzing infrastructure, the author of a fuzzer would need only provide a target fuzz function and how to build it to receive actionable bug reports in return.

This infrastructure can facilitate fuzzer execution and automate many fuzzer-related processes, such as:

• Retrieve and deploy the latest version of the code to be tested.
• Run fuzzers continuously.
• Synchronize corpora when fuzzers are run in parallel.
• Reproduce software defects caused by specific inputs.
• Minimize the number of inputs required to trigger specific software defects.
• Isolate the version in which a software defect is first observed.
• Open bug reports whenever an entry updates a new reproducible software defect.
• Close bug reports when a previously reproducible software defect is no longer observed.

Among the existing fuzzing infrastructures, there are:

OSS-Fuzz: It has found over 30,000 bugs in 500 open source projects. It supports:

• libFuzzer, AFL++, Honggfuzz in combination with Sanitizers.
• C/C++, Rust, Go, Python and Java/JVM code.
• x86_64 and i386 builds.

Syzkaller: An unsupervised, coverage-based kernel fuzzer. Initially developed with the Linux kernel in mind.

ClusterFuzzLite: A continuous fuzzing solution that runs as part of continuous integration (CI) workflows to find vulnerabilities faster. With a few lines of code, GitHub users can integrate ClusterFuzzLite into their workflow and fuzz pull requests to detect bugs before they are committed. Note: This one looks pretty awesome.

ClusterFuzz: A scalable fuzzing infrastructure used by Google to fuzz all Google products and as a fuzzing backend for OSS-Fuzz.