search

specron / framework

Lightweight, open source and magic-free framework for testing solidity smart contracts.
https://specron.github.io/framework
MIT License
35 stars 11 forks source link

Add coverage testing #79

Open fulldecent opened 4 years ago

fulldecent commented 4 years ago

This adds an option for coverage testing.

Thank you to Damiano Shehaj for help with this issue.

fulldecent commented 4 years ago

Pasting all the notes I have on this in case this sits for a while.

Damiano Shehaj 10:49 AM UTC, 24 Dec 2019 I mean your PR will work if you change this line. https://github.com/captainheart/specron-example/commit/49293ba097929c3f060975c330bc72c1126c0450

The gist is to let specron framework to test against coverage enabled ganache server of solidity-coverage.

This 'special' ganache server generates coverage report file during the test and it can be published to coveralls.io.

Why coveralls?

TLDR: We recommend Coveralls for the accuracy of its branch reporting.

https://github.com/sc-forks/solidity-coverage/blob/beta/docs/faq.md#continuous-integration

1. Changes for specron-framework

Most of my changes are in this commit https://github.com/captainheart/specron-framework/commit/ab992d12068e10c1f7cfe7b6297e5adb0f33e646

a. I added coverage arg to speocron-cli

https://github.com/captainheart/specron-framework/commit/ab992d12068e10c1f7cfe7b6297e5adb0f33e646#diff-49488acef9dcb4e8eb1de28c2fb5b602R89

b. If coverage arg is set, use new web3 provider.

https://github.com/captainheart/specron-framework/commit/ab992d12068e10c1f7cfe7b6297e5adb0f33e646#diff-048c4a4e54f5c1dc42069a0da927abdbR65

This provider points to coverage enabled ganache server of solidity-coverage which runs in background.

I used static member variable in order to preserve coverage setting (port number) in the class.

https://github.com/captainheart/specron-framework/commit/ab992d12068e10c1f7cfe7b6297e5adb0f33e646#diff-048c4a4e54f5c1dc42069a0da927abdbR23

If this is too hacky, we may read speocron prop of package.json and parse process.argv. Tried to minimize changes to the framework.

2. Changes for specron-example

This example requires specron cli installed globally. So we need Rush hack to use forked specron framework above.

a. Change folder structure to truffle's standard

b. Install solidity-coverage

Added .solcover.js

Added npm run test-coverage command

3. Next steps

There are interesting pre-release versions of solidity-coverage that supports Buidler Plugin and Coverage API. https://github.com/sc-forks/solidity-coverage/releases