search

project-zot / zot

zot - A scale-out production-ready vendor-neutral OCI-native container image/artifact registry (purely based on OCI Distribution Specification)
https://zotregistry.dev
Apache License 2.0
880 stars 93 forks source link

ci/cd: fuzz testing #351

Open rchincha opened 2 years ago

rchincha commented 2 years ago

Is your feature request related to a problem? Please describe.

Add fuzz-testing support

Describe the solution you'd like

Support was recently added to docker distribution. We can use that as a guideline.

https://github.com/google/oss-fuzz/tree/master/projects/distribution https://github.com/cncf/cncf-fuzzing/tree/main/projects/distribution

Additional context

https://en.wikipedia.org/wiki/Fuzzing

alexstan12 commented 2 years ago

At this stage, fuzzing complex fuzz targets seems to fail with no explanations, with log messages like: "fuzzing process hung or terminated unexpectedly: exit status 2" . When trying to reproduce using the test data generated by the fuzzer, the tests passed. Others have also experienced this, see https://faun.pub/best-practices-for-go-fuzzing-in-go-1-18-84eab46b70d8

Another problem occurs when fuzzer tries to minimize the so called failing data to a simpler form, that would still fail. The issue is described here https://github.com/golang/go/issues/52569 . Multiple seed corpuses were used, but the fuzz crash was the same.

https://github.com/alexstan12/zot/tree/fuzzing - here I wrote some fuzz tests for routes and storage_fs. Since simpler tests seem to work correctly ( eg. like those in storage_fs_test.go, that simply take a byte slice and pass it to the function to be fuzzed), I will focus on those.

NB: this was all done using native go fuzzing engine. The go_fuzz library still conflicts with 1.18 and couldn't be used. Issue described here https://github.com/dvyukov/go-fuzz/issues/338