search

OpenZeppelin / openzeppelin-test-environment

[Not actively maintained] One-line setup for blazing-fast smart contracts tests
https://docs.openzeppelin.com/test-environment
MIT License
90 stars 39 forks source link

Coverage script generates an invalid report with OZ Test Env + Chain Snapshots #169

Closed fforbeck closed 3 years ago

fforbeck commented 3 years ago

Since I've migrated from truffle tests to OZ Test env, and added evm_snapshot before the tests + evm_revert <snapshot-id> after each test case, the report of the code coverage does not show the correct coverage.

For instance:

  156 passing (6m)

---------------------------------|----------|----------|----------|----------|----------------|
File                             |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
---------------------------------|----------|----------|----------|----------|----------------|
 adapters/                       |        0 |        0 |        0 |    15.13 |                |
  BankAdapter.sol                |        0 |        0 |        0 |        0 |... 62,64,79,80 |
  Configuration.sol              |        0 |        0 |        0 |    29.41 |... 110,116,119 |
  CouponOnboarding.sol           |        0 |        0 |        0 |    23.81 |... 160,162,165 |
  DaoRegistryAdapter.sol         |        0 |      100 |        0 |        0 |          41,54 |
  Distribute.sol                 |        0 |        0 |        0 |     4.48 |... 338,341,342 |
  Financing.sol                  |        0 |        0 |        0 |    18.18 |... 125,130,133 |
  GuildKick.sol                  |        0 |        0 |        0 |    15.79 |... 145,147,153 |
  Managing.sol                   |        0 |        0 |        0 |       20 |... 130,132,138 |
  NFTAdapter.sol                 |        0 |      100 |        0 |        0 |       40,54,55 |
  Onboarding.sol                 |        0 |        0 |        0 |    18.46 |... 306,330,343 |
  Ragequit.sol                   |        0 |        0 |        0 |     12.5 |... 178,183,193 |
  Signatures.sol                 |        0 |        0 |        0 |    17.65 |... 115,120,124 |
  Tribute.sol                    |        0 |        0 |        0 |    15.63 |... 202,208,216 |
  TributeNFT.sol                 |        0 |        0 |        0 |       20 |... 172,177,189 |
 adapters/interfaces/            |      100 |      100 |      100 |      100 |                |
  IConfiguration.sol             |      100 |      100 |      100 |      100 |                |
  IDistribute.sol                |      100 |      100 |      100 |      100 |                |
  IFinancing.sol                 |      100 |      100 |      100 |      100 |                |
  IGuildKick.sol                 |      100 |      100 |      100 |      100 |                |
  IManaging.sol                  |      100 |      100 |      100 |      100 |                |
  IOnboarding.sol                |      100 |      100 |      100 |      100 |                |
  IRagequit.sol                  |      100 |      100 |      100 |      100 |                |
  ISignatures.sol                |      100 |      100 |      100 |      100 |                |
  IVoting.sol                    |      100 |      100 |      100 |      100 |                |
 adapters/voting/                |        0 |        0 |        0 |     1.46 |                |
  BatchVoting.sol                |        0 |        0 |        0 |     2.74 |... 359,360,363 |
  KickBadReporterAdapter.sol     |        0 |        0 |        0 |        0 |... 94,96,97,99 |
  OffchainVoting.sol             |        0 |        0 |        0 |     1.14 |... 700,703,705 |
  OffchainVotingHash.sol         |        0 |        0 |        0 |        0 |... 225,227,231 |
  SnapshotProposalContract.sol   |        0 |      100 |        0 |        0 |... 211,212,213 |
  Voting.sol                     |        0 |        0 |        0 |     3.13 |... 202,204,206 |
 core/                           |        0 |        0 |        0 |     1.79 |                |
  CloneFactory.sol               |        0 |      100 |        0 |        0 |    33,34,54,55 |
  DaoConstants.sol               |        0 |        0 |        0 |        0 |... 76,79,87,94 |
  DaoFactory.sol                 |        0 |        0 |        0 |     9.52 |... 161,162,167 |
  DaoRegistry.sol                |        0 |        0 |        0 |     0.74 |... 745,747,751 |
 extensions/                     |      100 |      100 |      100 |      100 |                |
  IExtension.sol                 |      100 |      100 |      100 |      100 |                |
 extensions/bank/                |        0 |        0 |        0 |     0.91 |                |
  Bank.sol                       |        0 |        0 |        0 |        0 |... 464,468,470 |
  BankFactory.sol                |        0 |      100 |        0 |       25 |       41,49,50 |
 extensions/erc1271/             |        0 |        0 |        0 |        0 |                |
  ERC1271.sol                    |        0 |        0 |        0 |        0 |... 102,107,120 |
  ERC1271Factory.sol             |        0 |      100 |        0 |        0 |       41,48,50 |
 extensions/executor/            |        0 |        0 |        0 |        0 |                |
  Executor.sol                   |        0 |        0 |        0 |        0 |... 138,146,154 |
  ExecutorFactory.sol            |        0 |      100 |        0 |        0 |       41,48,50 |
 extensions/nft/                 |        0 |        0 |        0 |      2.5 |                |
  NFT.sol                        |        0 |        0 |        0 |      2.7 |... 268,270,272 |
  NFTCollectionFactory.sol       |        0 |      100 |        0 |        0 |       40,47,48 |
 extensions/token/erc20/         |        0 |        0 |        0 |    10.13 |                |
  ERC20TokenExtension.sol        |        0 |        0 |        0 |     5.56 |... 347,350,353 |
  ERC20TokenExtensionFactory.sol |        0 |      100 |        0 |    57.14 |       39,51,52 |
 guards/                         |        0 |        0 |        0 |     9.09 |                |
  AdapterGuard.sol               |        0 |        0 |        0 |    16.67 |... 55,59,65,69 |
  MemberGuard.sol                |        0 |        0 |        0 |        0 |... 55,56,57,60 |
 helpers/                        |        0 |        0 |        0 |     5.26 |                |
  FairShareHelper.sol            |        0 |        0 |        0 |        0 |... 42,43,48,49 |
  GuildKickHelper.sol            |        0 |        0 |        0 |     7.69 |... 71,79,84,93 |
 test/                           |        0 |        0 |        0 |     3.57 |                |
  ERC20Minter.sol                |        0 |      100 |        0 |     12.5 |... 57,65,66,67 |
  OLToken.sol                    |        0 |      100 |        0 |        0 |             31 |
  PixelNFT.sol                   |        0 |        0 |        0 |        0 |... 80,82,84,93 |
  ProxToken.sol                  |        0 |      100 |        0 |        0 |    33,40,41,42 |
  TestFairShareCalc.sol          |        0 |      100 |        0 |        0 |             36 |
  TestToken1.sol                 |        0 |      100 |        0 |        0 |             32 |
  TestToken2.sol                 |        0 |      100 |        0 |        0 |             31 |
 utils/                          |        0 |        0 |        0 |      6.9 |                |
  DaoArtifacts.sol               |        0 |        0 |        0 |     12.5 |... 7,97,99,100 |
  Multicall.sol                  |        0 |        0 |        0 |        0 |... 52,60,64,68 |
  PotentialNewMember.sol         |        0 |        0 |        0 |       20 |    38,40,41,42 |
  Signatures.sol                 |        0 |      100 |        0 |        0 |       46,61,79 |
---------------------------------|----------|----------|----------|----------|----------------|
All files                        |        0 |        0 |        0 |     6.23 |                |
---------------------------------|----------|----------|----------|----------|----------------|

Any ideas on what could be causing this problem?

Before that change to OZ test env + Chain Snapshot the code coverage was ~80%.

Thanks

frangio commented 3 years ago

Are you sure this is a problem of Test Environment and not of solidity-coverage?

fforbeck commented 3 years ago

hi @frangio, not sure. I was wondering if due to the fact that the test env uses the runCoverage custom script to trigger the sol- cover I could be missing some configuration.