Add fuzzing by way of ClusterFuzzLite

[DavidKorczynski]

This adds fuzzing by way of ClusterFuzzLite, which is a GitHub action that will perform a short amount of fuzzing for new PRs. The goal is to use fuzzing to catch bugs that may be introduced by new PRs.

I added a fuzzer that targets reformat, and currently set the timeout of CFLite to 100 seconds. CFLite will flag if the fuzzer finds any issues in the code introduced by a PR.

To reproduce this set up the way ClusterFuzzLite does it (by way of OSS-Fuzz) you can do:

git clone https://github.com/google/oss-fuzz
git clone https://github.com/DavidKorczynski/json_struct
cd json_struct
git checkout clusterfuzzlite

# Build the fuzzers in .clusterfuzzlite
python3 ../oss-fuzz/infra/helper.py build_fuzzers --external $PWD

# Run the fuzzer for 10 seconds
python3 ../oss-fuzz/infra/helper.py run_fuzzer --external $PWD reformat_fuzzer-- -max_total_time=10

I think if you'd be happy to have this merged, then it would also make sense to adjust some of the asserts in the code to throw exceptions, since the fuzzer would be able to run for longer periods of time without interruption (the asserts cause the fuzzer to stop). There is a macro FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION which is enabled by default when compiling for fuzzing, so one option would be to add these macros around around the asserts to enforce such that the asserts are exceptions in fuzz mode, and then having the fuzzers catch these.

[jorgen]

Nice, I will review next time Im in front of a PC👍

[jorgen]

I have meant the asserts to be preconditions, and should be regarded as a bug if they are hit. I'm having some problem running the fuzzer since I only have a windows pc at the moment. I have docker, but there seems to to be an issue when running the docker image.

[jorgen]

Hi, I have now fixed the bugs found by address and fuzzy sanetizers. Thank you for making me aware of these problems and setting this up! I have also added the run to pushes to master, and added a badge to the readme. https://github.com/jorgen/json_struct/actions/workflows/cflite_pr.yml

[DavidKorczynski]

I'm having some problem running the fuzzer since I only have a windows pc at the moment. I have docker, but there seems to to be an issue when running the docker image.

I''m not so familiar with Windows for development, but I think the CFLite/OSS-Fuzz infra is not supported on Windows.

Thanks for updating here @jorgen -- looks great with the badge. A possible future option is to increase the seconds the fuzzer runs on each PR here https://github.com/jorgen/json_struct/blob/49d5cc660edca098383cf8343126069fbdba7991/.github/workflows/cflite_pr.yml#L29 The ClusterFuzzLite documentation suggests 600 seconds as the default.

[jorgen]

• Yeah sounds like a good idea. I ran it last night for several hours. Im going to enable asan on all tests on a builder. Would be cool if it was possible to get more close to json fuzzing. Ie json with random stuff inserted at random positions. Now it just fail after a few characters, and I would think not some more "complicated" states are triggered.