The way tests are implemented currently, it requires lots of manual changes to the CI config. Because of that, the CI configuration is getting bigger, noisy, harder to maintain and error-prone. It also contains lots of boilerplate to support cross-platform tests:
A simple option would be to move those commands into Shell scripts (Bash and PowerShell). This doesn't solve the problem of having lots of boilerplate but at least it makes the CI config less noisy (although at the cost of having more indirections).
Pros
✅ Less noise in the CI config
✅ Pretty simple shell scripting and not so much of the code if you're familiar with shell
✅ Testing CLI parsing as well
Cons
❌ Requires more polyglot knowledge
❌ Requires lots of manual updates
❌ Boilerplate
❌ Requires separate static analysis like shellcheck
Option 2: Rust integration tests
Alternatively, all these tests can be written as rust integration tests. They can be put into the tests/ directory and run as cargo test (which we already do)
Pros
✅ No polyglot env, only Rust for testing
✅ No boilerplate, only one test suite, reusing lib utils
✅ Automatically cross-platform
✅ All Rust tooling works for tests automatically
Cons
❌ More code
❌ No CLI parsing testing (maybe it can be done separately with clap?)
The way tests are implemented currently, it requires lots of manual changes to the CI config. Because of that, the CI configuration is getting bigger, noisy, harder to maintain and error-prone. It also contains lots of boilerplate to support cross-platform tests:
Just look at these 100 lines of YAML 😨
https://github.com/chshersh/tool-sync/blob/2bb99cf7724980f32e6ab1f970548b436aad99c3/.github/workflows/ci.yml#L43-L141
I propose
Option 1: Move to bash scripts
A simple option would be to move those commands into Shell scripts (Bash and PowerShell). This doesn't solve the problem of having lots of boilerplate but at least it makes the CI config less noisy (although at the cost of having more indirections).
Pros
✅ Less noise in the CI config ✅ Pretty simple shell scripting and not so much of the code if you're familiar with shell ✅ Testing CLI parsing as well
Cons
❌ Requires more polyglot knowledge ❌ Requires lots of manual updates ❌ Boilerplate ❌ Requires separate static analysis like
shellcheckOption 2: Rust integration tests
Alternatively, all these tests can be written as rust integration tests. They can be put into the
tests/directory and run ascargo test(which we already do)Pros
✅ No polyglot env, only Rust for testing ✅ No boilerplate, only one test suite, reusing
libutils ✅ Automatically cross-platform ✅ All Rust tooling works for tests automaticallyCons
❌ More code ❌ No CLI parsing testing (maybe it can be done separately with
clap?)