Open amusingaxl opened 4 months ago
What SHOULD be in tests:
file()
)What COULD be in tests:
What SHOULD NOT be in tests:
Special cases:
Agree in general, however:
What COULD be in tests:
This is not specific enough. I think our least responsibility is to ensure correct functioning of all systems after each spell. We don't write end-to-end tests for every single change (eg rate changes) only because either 1) we know there are no edge cases to hit there (or they otherwise checked within general tests / exec lib) or 2) end-to-end test is actually part of the general tests / required by the checklist (eg _checkIlkIntegration
). But for every new contract (except subdaos) I would always advice to have end-to-end happy path coverage.
Tests outside of the happy path (like the one prompted this discussion) can be clearly marked optional even if specifically requested by a reviewer.
it's expected that those are well documented and properly tested in the original repository.
Of course it's not expected that our review will magically replace in-depth audit. But I think it's spell reviewers responsibility to ensure (instead of expecting) that new contracts ether exactly match audits or (if there are no audits) they can not affect core functionality (e.g. does not get auth on vat). Not directly related to the test coverage discussion, we might want to clearly define this fine line. Or, if formulated differently: clearly limit what non-audited or even not-well-tested contract can do in a spell.
@SidestreamColdMelon please document the agreement on the Crafter / Reviewer checklists
Adding this as a placeholder so we can continue this discussion: https://github.com/makerdao/spells-mainnet/pull/417#discussion_r1691311398