search

OpenZeppelin / openzeppelin-test-helpers

Assertion library for Ethereum smart contract testing
https://docs.openzeppelin.com/test-helpers
MIT License
415 stars 132 forks source link

Error in `expectRevert` when used with Coveralls code coverage #164

Open David-Kneel opened 3 years ago

David-Kneel commented 3 years ago

When run yarn coveralls, the expectRevert command results in a failure.

❯ yarn coveralls
yarn run v1.22.10
$ yarn coverage && cat coverage/lcov.info | coveralls
$ truffle run coverage

> Using Truffle library from local node_modules.

> server:            http://127.0.0.1:8555
> truffle:           v5.3.6
> ganache-core:      v2.13.0
> solidity-coverage: v0.7.16

Network Info
============
> id:      *
> port:    8555
> network: soliditycoverage

Instrumenting for coverage...
=============================

> abdk-libraries-solidity/ABDKMath64x64.sol
> abdk-libraries-solidity/ABDKMathQuad.sol
> YieldFarming.sol
> YieldFarmingToken.sol

Coverage skipped for:
=====================

> Migrations.sol

Compiling your contracts...
===========================
✔ Fetching solc version list from solc-bin. Attempt #1
> Compiling ./.coverage_contracts/Migrations.sol
> Compiling ./.coverage_contracts/YieldFarming.sol
> Compiling ./.coverage_contracts/YieldFarmingToken.sol
> Compiling ./.coverage_contracts/abdk-libraries-solidity/ABDKMath64x64.sol
> Compiling ./.coverage_contracts/abdk-libraries-solidity/ABDKMathQuad.sol
> Compiling @openzeppelin/contracts/access/Ownable.sol
> Compiling @openzeppelin/contracts/token/ERC20/ERC20.sol
> Compiling @openzeppelin/contracts/token/ERC20/IERC20.sol
> Compiling @openzeppelin/contracts/token/ERC20/extensions/ERC20Burnable.sol
> Compiling @openzeppelin/contracts/token/ERC20/extensions/IERC20Metadata.sol
> Compiling @openzeppelin/contracts/token/ERC20/utils/SafeERC20.sol
> Compiling @openzeppelin/contracts/token/ERC20/utils/TokenTimelock.sol
> Compiling @openzeppelin/contracts/utils/Address.sol
> Compiling @openzeppelin/contracts/utils/Context.sol
✔ Fetching solc version list from solc-bin. Attempt #1
> Artifacts written to /home/daniel/Workspaces/Tachyonic/Personal/faster-than-light/.coverage_artifacts/contracts
> Compiled successfully using:
   - solc: 0.8.4+commit.c7e474f2.Emscripten.clang

Compiling your contracts...
===========================
✔ Fetching solc version list from solc-bin. Attempt #1
> Everything is up to date, there is nothing to compile.

  Contract: YieldFarming
    ✓ Ownership (136ms)
    Release token
      ✓ after unlock (2144ms)

  2 passing (3s)

------------------------------------|----------|----------|----------|----------|----------------|
File                                |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
------------------------------------|----------|----------|----------|----------|----------------|
 contracts/                         |      100 |       50 |      100 |      100 |                |
  YieldFarming.sol                  |      100 |       50 |      100 |      100 |                |
  YieldFarmingToken.sol             |      100 |      100 |      100 |      100 |                |
 contracts/abdk-libraries-solidity/ |      100 |      100 |    15.79 |    15.79 |                |
  ABDKMath64x64.sol                 |      100 |      100 |        0 |        0 |... 659,678,728 |
  ABDKMathQuad.sol                  |      100 |      100 |       30 |       30 |... 66,990,1198 |
------------------------------------|----------|----------|----------|----------|----------------|
All files                           |      100 |       50 |       25 |    33.33 |                |
------------------------------------|----------|----------|----------|----------|----------------|

> Istanbul reports written to ./coverage/ and ./coverage.json
> solidity-coverage cleaning up, shutting down ganache server
Done in 28.92s.
❯ yarn coveralls
yarn run v1.22.10
$ yarn coverage && cat coverage/lcov.info | coveralls
$ truffle run coverage

> Using Truffle library from local node_modules.

> server:            http://127.0.0.1:8555
> truffle:           v5.3.6
> ganache-core:      v2.13.0
> solidity-coverage: v0.7.16

Network Info
============
> id:      *
> port:    8555
> network: soliditycoverage

Instrumenting for coverage...
=============================

> abdk-libraries-solidity/ABDKMath64x64.sol
> abdk-libraries-solidity/ABDKMathQuad.sol
> YieldFarming.sol
> YieldFarmingToken.sol

Coverage skipped for:
=====================

> Migrations.sol

Compiling your contracts...
===========================
✔ Fetching solc version list from solc-bin. Attempt #1
> Compiling ./.coverage_contracts/Migrations.sol
> Compiling ./.coverage_contracts/YieldFarming.sol
> Compiling ./.coverage_contracts/YieldFarmingToken.sol
> Compiling ./.coverage_contracts/abdk-libraries-solidity/ABDKMath64x64.sol
> Compiling ./.coverage_contracts/abdk-libraries-solidity/ABDKMathQuad.sol
> Compiling @openzeppelin/contracts/access/Ownable.sol
> Compiling @openzeppelin/contracts/token/ERC20/ERC20.sol
> Compiling @openzeppelin/contracts/token/ERC20/IERC20.sol
> Compiling @openzeppelin/contracts/token/ERC20/extensions/ERC20Burnable.sol
> Compiling @openzeppelin/contracts/token/ERC20/extensions/IERC20Metadata.sol
> Compiling @openzeppelin/contracts/token/ERC20/utils/SafeERC20.sol
> Compiling @openzeppelin/contracts/token/ERC20/utils/TokenTimelock.sol
> Compiling @openzeppelin/contracts/utils/Address.sol
> Compiling @openzeppelin/contracts/utils/Context.sol
✔ Fetching solc version list from solc-bin. Attempt #1
> Artifacts written to /home/daniel/Workspaces/Tachyonic/Personal/faster-than-light/.coverage_artifacts/contracts
> Compiled successfully using:
   - solc: 0.8.4+commit.c7e474f2.Emscripten.clang

Compiling your contracts...
===========================
✔ Fetching solc version list from solc-bin. Attempt #1
> Everything is up to date, there is nothing to compile.

  Contract: YieldFarming
    ✓ Ownership (200ms)
    Release token
      1) before unlock

    Events emitted during test:
    ---------------------------

    Ownable.OwnershipTransferred(
      previousOwner: <indexed> 0x0000000000000000000000000000000000000000 (type: address),
      newOwner: <indexed> 0x8C62793E27cf14e3074be39817F84faabc39c8be (type: address)
    )

    Ownable.OwnershipTransferred(
      previousOwner: <indexed> 0x0000000000000000000000000000000000000000 (type: address),
      newOwner: <indexed> 0x6a4F343faC135136b6482B280df8305B03018F51 (type: address)
    )

    IERC20.Transfer(
      from: <indexed> 0x0000000000000000000000000000000000000000 (type: address),
      to: <indexed> 0x6d0982d6A0f46aE78141cDB761a5455241D4812f (type: address),
      value: 14426950408889 (type: uint256)
    )

    ---------------------------
      ✓ after unlock (2141ms)

  2 passing (5s)
  1 failing

  1) Contract: YieldFarming
       Release token
         before unlock:
     Error: Invalid JSON RPC response: ""
      at Object.InvalidResponse (node_modules/@openzeppelin/test-helpers/node_modules/web3-core-helpers/lib/errors.js:43:16)
      at XMLHttpRequest.request.onreadystatechange (node_modules/@openzeppelin/test-helpers/node_modules/web3-providers-http/lib/index.js:95:32)
      at XMLHttpRequestEventTarget.dispatchEvent (node_modules/xhr2-cookies/xml-http-request-event-target.ts:44:13)
      at XMLHttpRequest._setReadyState (node_modules/xhr2-cookies/xml-http-request.ts:219:8)
      at XMLHttpRequest._onHttpRequestError (node_modules/xhr2-cookies/xml-http-request.ts:379:8)
      at ClientRequest.<anonymous> (node_modules/xhr2-cookies/xml-http-request.ts:266:37)
      at Socket.socketErrorListener (_http_client.js:427:9)
      at emitErrorNT (internal/streams/destroy.js:92:8)
      at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
      at processTicksAndRejections (internal/process/task_queues.js:84:21)

------------------------------------|----------|----------|----------|----------|----------------|
File                                |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
------------------------------------|----------|----------|----------|----------|----------------|
 contracts/                         |      100 |       50 |      100 |      100 |                |
  YieldFarming.sol                  |      100 |       50 |      100 |      100 |                |
  YieldFarmingToken.sol             |      100 |      100 |      100 |      100 |                |
 contracts/abdk-libraries-solidity/ |      100 |      100 |    15.79 |    15.79 |                |
  ABDKMath64x64.sol                 |      100 |      100 |        0 |        0 |... 659,678,728 |
  ABDKMathQuad.sol                  |      100 |      100 |       30 |       30 |... 66,990,1198 |
------------------------------------|----------|----------|----------|----------|----------------|
All files                           |      100 |       50 |       25 |    33.33 |                |
------------------------------------|----------|----------|----------|----------|----------------|

> Istanbul reports written to ./coverage/ and ./coverage.json
> solidity-coverage cleaning up, shutting down ganache server
Error: ❌ 1 test(s) failed under coverage.
    at plugin (/home/daniel/Workspaces/Tachyonic/Personal/faster-than-light/node_modules/solidity-coverage/plugins/truffle.plugin.js:121:27)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
Truffle v5.3.6 (core: 5.3.6)
Node v12.22.1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

This situation is evidenced in the following GitHub actions run

frangio commented 3 years ago

Are yous ure this is happening in expectRevert? I don't see how this could be caused by test-helpers.

David-Kneel commented 3 years ago

Are yous ure this is happening in expectRevert? I don't see how this could be caused by test-helpers.

If you observe int the CI/CD run, all tests pass when run by command yarn test but the only one test with expectRevert fails if tests are triggered by yarn coverage .

I have no indication whether the problem lies in OpenZeppelin test helpers or in npm package solidity-coverage.