supranational / blst

Multilingual BLS12-381 signature library
Apache License 2.0
458 stars 175 forks source link

.github/workflows/: add CIFuzz action #141

Closed DavidKorczynski closed 1 year ago

DavidKorczynski commented 1 year ago

Add CIFuzz workflow action to have fuzzers build and run on each PR.

This is a service offered by OSS-Fuzz where blst already runs (https://github.com/google/oss-fuzz/tree/master/projects/bls-signatures). CIFuzz can help detect catch regressions and fuzzing build issues early, and has a variety of features (see the URL above). In the current PR the fuzzers gets build on a pull request and will run for 300 seconds.

Signed-off-by: David Korczynski david@adalogics.com

dot-asm commented 1 year ago

Well, the action failed, and for unrelated reasons. Which is unsustainable and formally disqualifying. But even if it worked [reliably], one can still question its utility. As follows. The only code that is exercised by OSS-Fuzz is one in the src directory. But we know that there won't be any meaningful changes to it. Not to the parts that are fuzzed. With this in mind, what would the action's success or failure mean? I would argue that it would rather vouch (or not) for the compiler than blst. In other words success would tell virtually nothing about blst per se, while failure would be a distraction.

The above is not to say "just no," but that we have to understand the limitations. Meanwhile suggestion would be to add path: directive and limit the action's trigger to modifications in src directory [and the cifuzz.yml itself]. Next question would be if there are stable versions, as opposed to presumably bleeding-edge @master tag, which apparently is not as reliable as one would desire.

As for "understanding the limitations." It's rather about value vs. cost. As for the latter. Even if it doesn't cost anything to us, it doesn't mean that it has no cost. And it would be inappropriate to do things on "why-not" basis ;-)