Closed masih closed 5 months ago
We should first add test coverage to CI! @masih would you look into this?
I predict this will show some holes in our unit testing at the level of explicitly testing message sequences given to the participant. This is a gap between the simple data structure unit tests and sim-level exercises we currently have.
Ah, actually, go fuzzing already works this way. It just stores the entries in $(go env GOCACHE)/fuzz
. We need to upload this somewhere when the test finishes (success or failure) and download it afterwards.
We could use caching but... ideally we'd push it to a git repo somewhere. @Kubuxu should be able to help here.
We should first add test coverage to CI! @masih would you look into this?
Will do; that should be quick.
ideally we'd push it to a git repo somewhere
I would just push those to testdata
next to the tests. That would be easier; no?
It might be more optimal to move away from seed fuzzing and instead fuzze an array of latencies or something similar, but seed-based fuzzing is a great initial step.
Giving more precise control over latencies to the fuzzer would make it more expressive, allowing it to test more scenarios and allow it to make smaller changes to the scenario as it evolves.
We could use caching but... ideally we'd push it to a git repo somewhere. @Kubuxu should be able to help here.
In my opinion, we should store minimized fuzz corpus in this repo. The separate repo works but is a bit hacky, and I've done it in the past mostly because the fuzzer used was AFL, and I was fuzzing complex scenarios with large churn.
Hmm, looks like go doesn't support corpus minimization right now :/
-fuzzminimizetime
not it?
Nope, that is for minimizing the crash reproducer (essentially, it will take the crasher and fuzz it to reduce its size while keeping the crash).
I will close this as Golang's fuzzing is coverage-guided. For a more advanced fuzzing strategy (message reordering), we might want to open a new issue.
Also see #218
It would be really nice if we could have coverage guided fuzzing.
We'll still want random fuzzing, but this will help us make sure we at least cover as many error paths as possible.
Originally posted by @Stebalien in https://github.com/filecoin-project/go-f3/issues/219#issuecomment-2111064226