cooklang / cooklang-rs

Canonical Cooklang parser in Rust
https://cooklang.github.io/cooklang-rs/
MIT License
44 stars 6 forks source link

Add fuzzing for the rust parser #9

Closed stusmall closed 10 months ago

stusmall commented 10 months ago

This adds a fuzzing task for the rust parser. This is run as release to avoid various debug assertions in the project. I've had this running on my workstation and it hasn't found anything, but it is useful to keep in CI to catch regressions.

I recongize that fuzzing adds extra time to the pipeline, so I did a few things to try and minimize the impact.

Later I will take a look at having it excercise more parser configuration options to get better coverage. At first I just wanted to try and cover the default case well.

dubadub commented 10 months ago

Looks good to me, what do you think @Zheoni?

Zheoni commented 10 months ago

I'm not familiar with fuzz testing. I understand that it tries random inputs to check that the parser doesn't panic, is that correct?

If that's it I'm happy to merge this, caching the builds it's great too! 😄

stusmall commented 10 months ago

Yup. The idea is that it will generate random input and observes coverage. It mutates the input to try and maximize coverage. It's really useful for things like this that take broad, plaintext input and parses it. It helps shake out hidden panics, logic bugs, unicode mistakes, infinite loops and various crashes.

Fuzzers are generally fairly dumb and don't give you a lot of input on if the parsing was "correct", just that it didn't explode

Zheoni commented 10 months ago

Ok, seems like a good idea to have it. Thanks!