Closed DaniPopes closed 1 week ago
I am applying to this issue via OnlyDust platform.
I am a security researcher so I have some experience with testing
I will follow your clear instructions and make the required changes to make the testing more scalable
Currently our CLI/UI tests are doing manual assertions with the output, manually parsing JSON and comparing against another manually parsed fixture file etc.
This is not ideal, and not very scalable as any change in the output will also require manually updating all the fixture files.
We should migrate to using the
snapbox
library which is also used bycargo
in most of their tests.Supersedes https://github.com/foundry-rs/foundry/issues/7604
Unblocks https://github.com/foundry-rs/foundry/pull/6569
Completed in #8406
Migration guide
https://github.com/foundry-rs/foundry/pull/8406 added helper
assert()
method toTestCommand
, which executes configured command and returns snapbox'sOutputAssert
allowing to perform checks on output.For tests which perform assertions on stdout, instead of asserting value of
stdout_lossy
/unchecked_stdout
prefer usingassert().stdout_eq(str![["<fixture>"]])
orassert_success().stdout_eq(str![["<fixture>"]])
if you need to assert successful status code.i.e the following test
would become
For tests using fixture files, prefer inlining those fixtures into
str!
macro invocations and removing the fixture files.i.e such assertion
would become
Large fixture files can be kept as is, prefer using
file!
macro for those.Prefer replacing all irrelevant info in fixtures with snapbox filters (
...
as line wildcard,[..]
as character-wildcard). i.e. if we're asserting on traces and output looks like this:the test would look similar to:
Note: when writing tests it might be handy to set fixture to empty value (
str![[""]]
) and populate by runningcargo test
withSNAPSHOTS=overwrite
to let snapbox populate fixtures automatically