foundry-rs / foundry

Foundry is a blazing fast, portable and modular toolkit for Ethereum application development written in Rust.
https://getfoundry.sh
Apache License 2.0
8.1k stars 1.67k forks source link

meta(`testing`): migrate CLI / UI tests to `snapbox` #8389

Closed DaniPopes closed 1 week ago

DaniPopes commented 2 months ago

Currently our CLI/UI tests are doing manual assertions with the output, manually parsing JSON and comparing against another manually parsed fixture file etc.

This is not ideal, and not very scalable as any change in the output will also require manually updating all the fixture files.

We should migrate to using the snapbox library which is also used by cargo in most of their tests.

Supersedes https://github.com/foundry-rs/foundry/issues/7604

Unblocks https://github.com/foundry-rs/foundry/pull/6569

The migration can be done gradually over multiple commits, however we also need an initial integration with the TestCommand to allow asserting with snapbox's str! and file! macros.

Completed in #8406

Migration guide

https://github.com/foundry-rs/foundry/pull/8406 added helper assert() method to TestCommand, which executes configured command and returns snapbox's OutputAssert allowing to perform checks on output.

0xjarix commented 1 month ago

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

I am a security researcher so I have some experience with testing

How I plan on tackling this issue

I will follow your clear instructions and make the required changes to make the testing more scalable