Closed 0xean closed 1 month ago
App | Unit tests | Integration tests | Blockchain simulation | |
---|---|---|---|---|
1 | Uniswap | Jest | Cypress | Hardhat |
2 | Pancake | Jest | Cypress | None |
3 | Jumper | None | None | None |
4 | Spectrum | None | None | None |
5 | Bancor | vitest | Playwrite | Tenderly |
I've reviewed a number of open source DEX repositories that have similar functionality to us, namely those that include a swapper.
Uniswap and Pancake used Cypress for their end to end integration tests. A standout was Uniswap, which seems the most extensive, as it includes the ability to simulate and mock on-chain behaviour via Hardhat.
Additionally, Uniswap:
eth_sendRawTransaction
) are called at the expected timescypress-hardhat
, which they also maintain, to make Cypress play with HardhatWe have had a Cypress implementation before, though it never got past the PoC phase. For this reason, the maintenance burden and CI cost made it difficult to justify keeping.
It did basic checks, including:
It did not do any checks surrounding wallet methods or on-chain activity.
Implement Cypress for end-to-end integration testing, configured to use Hardhat forks across a 2-3 chains.
For this to be useful and worth doing we'll want to ensure that we include sufficient test cases for our revenue-driving feature-set. For example, the below checks should be feasible with this architecture:
Swapper
LP
Sends
Savers
Performance (network requests)
Use integration tests to check for network performance regressions (an increase in the number of calls to expensive endpoints caused by unoptimised reactivity, for example)
Excerpt from Uniswap README:
Our tests use a local hardhat node to simulate blockchain transactions. This can be accessed with `cy.hardhat().then((hardhat) => ...)`.
By default, automining is turned on, so that any transaction that you send to the blockchain is mined immediately. If you want to assert on intermediate states (between sending a transaction and mining it), you can turn off automining: `cy.hardhat({ automine: false })`.
The hardhat integration has built-in utilities to let you modify and assert on balances, approvals, and permits, and should be fully typed. Check it out at [Uniswap/cypress-hardhat](https://github.com/Uniswap/cypress-hardhat).
Asserting on wallet methods:
// Asserts that `eth_sendRawTransaction` was sent to the wallet.
cy.wait('@eth_sendRawTransaction')
Sometimes, you may want a method to fail. In this case, you can stub it, but you should disable logging to avoid spamming the test:
// Stub calls to eth_signTypedData_v4 and fail them
cy.hardhat().then((hardhat) => {
// Note the closure to keep signTypedDataStub in scope. Using closures instead of variables (eg let) helps prevent misuse of chaining.
const signTypedDataStub = cy.stub(hardhat.provider, 'send').log(false)
signTypedDataStub.withArgs('eth_signTypedData_v4).rejects(USER_REJECTION)
signTypedDataStub.callThrough() // allws other methods to call through to hardhat
cy.contains('Confirm swap').click()
// Verify the call occured
// Note the call to cy.wrap to correctly queue the chained command. Without this, the test would occur before the stub is called.
cy.wrap(permitApprovalStub).should('be.calledWith', 'eth_signTypedData_v4')
// Restore the stub
// note the call to cy.then to correctly queue the chained command. Without this, the stub would be restored immediately.
cy.then(() => permitApprovalStub.restore())
})
Overview
Currently our process for validating changes to the code base heavily relies on manual testing against production blockchains. This is slow, error prone, and can be expensive.
testnet solutions don't fully resolve any of these pain points due to the cross chain nature (EVM <> BTC) of some of our most critical products.
Additionally the number or protocols x number of wallets x number of chains will continue to grow simply making the problem worse.
We should evaluate if other solutions or architectures might prove effective to better validate our code or the ability to implement more automated testing. Another option might be to establish more formal and documented test plans that engineering sticks to when validating changes.
References and additional details
Lets timebox this to 2 days of effort for the moment, if we need to invest more time we can but lets a tleast check in at that point.
Acceptance Criteria
possible sources of information:
at the end, generally summarize findings and proposed next steps either in this ticket or another doc.
Need By Date
No response
Screenshots/Mockups
No response
Estimated effort
2