npx hardhat coverage fails - stack too deep

mreitschuster commented 1 month ago

npx hardhat coverage --show-stack-traces

Version
=======
> solidity-coverage: v0.8.12

Instrumenting for coverage...
=============================

> APIConsumer.sol
> AutomationCounter.sol
> PriceConsumerV3.sol
> RandomNumberConsumerV2Plus.sol
> RandomNumberDirectFundingConsumerV2Plus.sol
> test/VRFCoordinatorV2_5Mock.sol
> VRFV2PlusWrapper.sol

Coverage skipped for:
=====================

> test/fuzzing/AutomationCounterEchidnaTest.sol
> test/MockLinkToken.sol
> test/MockOracle.sol
> test/MockV3Aggregator.sol

Compilation:
============

CompilerError: Stack too deep. Try compiling with `--via-ir` (cli) or the equivalent `viaIR: true` (standard JSON) while enabling the optimizer. Otherwise, try removing local variables. When compiling inline assembly: Variable headStart is 1 slot(s) too deep inside the stack. Stack too deep. Try compiling with `--via-ir` (cli) or the equivalent `viaIR: true` (standard JSON) while enabling the optimizer. Otherwise, try removing local variables.

after checking out, npm install, npx hardhat compile, npx hardhat test, I am trying to get the code coverage. It fails with stack too deep. In order to solve the above issue, I tinkered around with

enabling the via-ir flag
excluding more/all of the .sol files in .solcover.js
adding a @skip-on-coverage to the test definitions ( it("testname [ @skip-on-coverage ]", async () => {}))

but with none of those i was able to get code coverage running

Any hint is appreciated to help me understand how to get code coverage for the code to run.

zeuslawyer commented 1 month ago

Hey @mreitschuster Thanks for raising this. I was able to reproduce. The fix appears to be adding configureYulOptimizer: true to .solcover.js.

Try that out and let us know if its working? Note that npm run tests will work but npm run coverage may cause some failures in the unit test for VRF Direct Funding. We are looking into that too.

mreitschuster commented 1 month ago

thank you @zeuslawyer works like a charm.

mreitschuster commented 1 month ago

sorry. too eager too early. the error "stack too deep" is now gone. but there is another error now - not sure if this belongs in the same or in a different issue: npx hardhat test works fine, but npx hardhat coverage fails some tests


npx hardhat test
WARNING: You are currently using Node.js v22.4.1, which is not supported by Hardhat. This can lead to unexpected behavior. See https://hardhat.org/nodejs-versions

  Automation Counter Unit Tests
    #checkUpkeep
      success
        ✔ should be able to call checkUpkeep (242ms)
    #performUpkeep
      success
        ✔ should be able to call performUpkeep after time passes
      failure
        ✔ should not be able to call perform upkeep without the time passed interval

  Price Consumer Unit Tests
    deployment
      success
        ✔ should set the aggregator addresses correctly (69ms)
    #getLatestPrice
      success
        ✔ should return the same value as the mock

  Random Number Consumer Unit Tests
    #requestRandomWords
      success
        ✔ Should successfully request a random number (116ms)
        ✔ Should successfully request a random number and get a result (59ms)
ReturnedRandomness event fired!
        ✔ Should successfully fire event on callback (4029ms)

  Random Number Direct Funding Consumer Unit Tests
    #requestRandomWords
      success
        ✔ Should successfully request a random number (216ms)
price 2683220000000000000
paid 2683220000000000000
wrapper balance 2683220000000000000
initial subscription balance 100000000000000000000
final subscription balance 99294505000000000000
paid by wrapper 705495000000000000
wrapperPremium 1977725000000000000
percentage 73
        ✔ Should successfully request a random number and get a result (95ms)
        ✔ Should be able to request several random words (92ms)
      error
        ✔ Cannot request randomness without funding wrapper
        ✔ Cannot request more than maxNumWords  (46ms)
        ✔ Cannot fulfill if callback gasLimit too low  (60ms)

  14 passing (5s)


npx hardhat coverage
WARNING: You are currently using Node.js v22.4.1, which is not supported by Hardhat. This can lead to unexpected behavior. See https://hardhat.org/nodejs-versions

Version
=======
> solidity-coverage: v0.8.12

Instrumenting for coverage...
=============================

> AutomationCounter.sol
> PriceConsumerV3.sol
> RandomNumberConsumerV2Plus.sol
> RandomNumberDirectFundingConsumerV2Plus.sol
> test/VRFCoordinatorV2_5Mock.sol
> VRFV2PlusWrapper.sol

Coverage skipped for:
=====================

> test/fuzzing/AutomationCounterEchidnaTest.sol
> test/MockLinkToken.sol
> test/MockV3Aggregator.sol

Compilation:
============

Compiled 40 Solidity files successfully (evm target: paris).

Network Info
============
> HardhatEVM: v2.22.3
> network:    hardhat

  Automation Counter Unit Tests
    #checkUpkeep
      success
        ✔ should be able to call checkUpkeep (59ms)
    #performUpkeep
      success
        ✔ should be able to call performUpkeep after time passes
      failure
        ✔ should not be able to call perform upkeep without the time passed interval

  Price Consumer Unit Tests
    deployment
      success
        ✔ should set the aggregator addresses correctly (88ms)
    #getLatestPrice
      success
        ✔ should return the same value as the mock

  Random Number Consumer Unit Tests
    #requestRandomWords
      success
        ✔ Should successfully request a random number (99ms)
        ✔ Should successfully request a random number and get a result (39ms)
ReturnedRandomness event fired!
        ✔ Should successfully fire event on callback (4032ms)

  Random Number Direct Funding Consumer Unit Tests
    #requestRandomWords
      success
        1) Should successfully request a random number
        2) Should successfully request a random number and get a result
        3) Should be able to request several random words
      error
        4) Cannot request randomness without funding wrapper
        5) Cannot request more than maxNumWords 
        6) Cannot fulfill if callback gasLimit too low 

  8 passing (5s)
  6 failing

  1) Random Number Direct Funding Consumer Unit Tests
       #requestRandomWords
         success
           Should successfully request a random number:
     ProviderError: Sender doesn't have enough funds to send tx. The max upfront cost is: 109951064975100000000000 and the sender's balance is: 9999999999999998471072.
      at EdrProviderWrapper.request (/home/username/src/hardhat-starter-kit/node_modules/hardhat/src/internal/hardhat-network/provider/provider.ts:430:19)
      at EthersProviderWrapper.send (/home/username/src/hardhat-starter-kit/node_modules/@nomiclabs/hardhat-ethers/src/internal/ethers-provider-wrapper.ts:13:20)

  2) Random Number Direct Funding Consumer Unit Tests
       #requestRandomWords
         success
           Should successfully request a random number and get a result:
     ProviderError: Sender doesn't have enough funds to send tx. The max upfront cost is: 109951064975100000000000 and the sender's balance is: 9999999999999998471072.
      at EdrProviderWrapper.request (/home/username/src/hardhat-starter-kit/node_modules/hardhat/src/internal/hardhat-network/provider/provider.ts:430:19)
      at EthersProviderWrapper.send (/home/username/src/hardhat-starter-kit/node_modules/@nomiclabs/hardhat-ethers/src/internal/ethers-provider-wrapper.ts:13:20)

  3) Random Number Direct Funding Consumer Unit Tests
       #requestRandomWords
         success
           Should be able to request several random words:
     ProviderError: Sender doesn't have enough funds to send tx. The max upfront cost is: 109951064975100000000000 and the sender's balance is: 9999999999999998471072.
      at EdrProviderWrapper.request (/home/username/src/hardhat-starter-kit/node_modules/hardhat/src/internal/hardhat-network/provider/provider.ts:430:19)
      at EthersProviderWrapper.send (/home/username/src/hardhat-starter-kit/node_modules/@nomiclabs/hardhat-ethers/src/internal/ethers-provider-wrapper.ts:13:20)

  4) Random Number Direct Funding Consumer Unit Tests
       #requestRandomWords
         error
           Cannot request randomness without funding wrapper:
     ProviderError: Sender doesn't have enough funds to send tx. The max upfront cost is: 109951064975100000000000 and the sender's balance is: 9999999999999998471072.
      at EdrProviderWrapper.request (/home/username/src/hardhat-starter-kit/node_modules/hardhat/src/internal/hardhat-network/provider/provider.ts:430:19)
      at EthersProviderWrapper.send (/home/username/src/hardhat-starter-kit/node_modules/@nomiclabs/hardhat-ethers/src/internal/ethers-provider-wrapper.ts:13:20)

  5) Random Number Direct Funding Consumer Unit Tests
       #requestRandomWords
         error
           Cannot request more than maxNumWords :
     ProviderError: Sender doesn't have enough funds to send tx. The max upfront cost is: 109951064975100000000000 and the sender's balance is: 9999999999999998471072.
      at EdrProviderWrapper.request (/home/username/src/hardhat-starter-kit/node_modules/hardhat/src/internal/hardhat-network/provider/provider.ts:430:19)
      at EthersProviderWrapper.send (/home/username/src/hardhat-starter-kit/node_modules/@nomiclabs/hardhat-ethers/src/internal/ethers-provider-wrapper.ts:13:20)

  6) Random Number Direct Funding Consumer Unit Tests
       #requestRandomWords
         error
           Cannot fulfill if callback gasLimit too low :
     ProviderError: Sender doesn't have enough funds to send tx. The max upfront cost is: 109951064975100000000000 and the sender's balance is: 9999999999999998471072.
      at EdrProviderWrapper.request (/home/username/src/hardhat-starter-kit/node_modules/hardhat/src/internal/hardhat-network/provider/provider.ts:430:19)
      at EthersProviderWrapper.send (/home/username/src/hardhat-starter-kit/node_modules/@nomiclabs/hardhat-ethers/src/internal/ethers-provider-wrapper.ts:13:20)

----------------------------------------------|----------|----------|----------|----------|----------------|
File                                          |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
----------------------------------------------|----------|----------|----------|----------|----------------|
 contracts/                                   |       30 |    21.43 |    66.67 |       50 |                |
  AutomationCounter.sol                       |      100 |      100 |      100 |      100 |                |
  PriceConsumerV3.sol                         |      100 |      100 |      100 |      100 |                |
  RandomNumberConsumerV2Plus.sol              |      100 |       50 |      100 |      100 |                |
  RandomNumberDirectFundingConsumerV2Plus.sol |        0 |        0 |    16.67 |     5.26 |... 119,126,127 |
  VRFV2PlusWrapper.sol                        |      100 |      100 |      100 |      100 |                |
 contracts/test/                              |      100 |      100 |      100 |      100 |                |
  VRFCoordinatorV2_5Mock.sol                  |      100 |      100 |      100 |      100 |                |
----------------------------------------------|----------|----------|----------|----------|----------------|
All files                                     |       30 |    21.43 |    66.67 |       50 |                |
----------------------------------------------|----------|----------|----------|----------|----------------|

> Istanbul reports written to ./coverage/ and ./coverage.json
Error in plugin solidity-coverage: ❌ 6 test(s) failed under coverage.

For more info run Hardhat with --show-stack-traces

zeuslawyer commented 1 month ago

@mreitschuster thats right - i had called that out in my comment above :) looking into that.

smartcontractkit / hardhat-starter-kit

npx hardhat coverage fails - stack too deep #190