Incorporate the dump of all deployed contracts into tests

Zilliqa / scilla

Scilla - A Smart Contract Intermediate Level Language

https://scilla-lang.org

GNU General Public License v3.0

240 stars 79 forks source link

Incorporate the dump of all deployed contracts into tests #1120

Open anton-trunov opened 2 years ago

anton-trunov commented 2 years ago

There are around 1000 more or less unique contracts deployed on the Zilliqa blockchain as of time of creating this issue. We should incorporate these into our test suite in some form.

Some thoughts on this issue:

it seems valuable to manually analyse several tens of most active (popular) contracts and make more thorough tests out of those;
it would be great to record contract states and transitions and put them into the scilla-runner tests, ensuring we can replicate real-world contracts' behaviors when improving or fixing Scilla.

jubnzv commented 1 year ago

I'm a little worried about possible copyright issues when storing these contracts in our test suite. And it will also significantly increase the size of the repository (so CI will be slower and it may be inconvenient for users to clone the repository).

I think another use case for such a test is to detect false positives/false negatives of a static analyzer. This could be done as a separate script, that could be run manually (or via CI), which shows, how the static analyzer warnings on a contract base are changed between two commits. We could use it before releasing a new version of Scilla or create some integration with CI, that prints changes in warnings between master and the PR branch.

What do you think?

rrw-zilliqa commented 1 year ago

@jubnzv - I agree. We definitely oughtn't to store these in the test suite - it'll be enormous - but a tool to extract transitions from persistence and run the test might be useful (if we want reference data, we can store this hashed to avoid doing any copying).

Agreed re static analyzer - worth keeping in mind for when that is a possibility.

Seem fair?

anton-trunov commented 1 year ago

Just to give more context on this:

There are 970 unique Scilla contracts in the contract dump
It's 428743 lines of code in total (average contract size is about 440 LoC)
it's 15Mb of text

A lot of contracts are very similar with a diff of only around 10 lines or so.

jjcnn commented 1 year ago

No chance of having this ready in time for v0.13. Moving to v0.14.