Port over pyethereum to Py-EVM/Web3.py/eth-tester

[fubuloubu]

What's your issue about?

@pipermerriam @carver are working hard on creating the Python infrastructure to replace pyethereum. Would be great to port our testing infrastructure over to leverage what they're creating with Py-EVM and eth-tester (facilitated thru Web3.py).

How can it be fixed?

Personally, I have some scrawlings of a py.test plugin (pytest-ethereum) that leverages the ethereum python ecosystem to provide the necessary fixtures needed to test contracts (mostly stuff we have in our own conftest.py), including in-place source code compilation to contract class (leveraging the ImplicitContract class I added to Web3.py). The hope is that we don't have to change too much about our tests after moving to these set of tools.

I'd like this plugin to transfer into a more standardized way to create functional testing suites in a python environment, so this would be a good challenge to help that tool get there (testing with near-zero overhead = :+1: :+1:).

Let me know your thoughts in general about conversion, and specifically about my tool.

[carver]

Cool, an easy-to-spin up ethereum python test fixture would be super useful!

I'm not a fan of the approach of using a custom config yml and json files to define the setup (please correct me if I'm misinterpreting the examples folder in the project you linked). I think it should all be plain python or use the ethpm spec https://github.com/ethpm/ethpm-spec

[fubuloubu]

@carver

For contracts.json, ethpm might be a good way to go. I am not too familar with it to make a determination of whether it suits the needs of handling the compiler artifacts locally. Please check out #642, we were discussing something very similar, so there's probably some synergies we can apply to the overall Ethereum ecosystem in handing compiler artifacts (as well as using them in tools) because package management has similar goals. I wanted to leverage the artifact outputs from the compiler so the tools would remain language agnostic, making it trivial to add new language support. There could then create an eth-compilers module that aggregates all the language-specific efforts of generating these artifacts from a folder listing, keeping track of their status via a source-code checksum and outputting standardized artifacts.

For config.yml, I wanted a way to define the default deployment arguments for generating fixtures in a very clear, concise format (without having to write more custom python code) because the code for getting a contract instance from web3 is fairly verbose. I know you guys like to avoid "magic", but for the majority of contract testing, I basically want to get directly to the testing part without having to maintain extra configuration in python. This is why I added that ImplicitContract class, so that testing would be as succinct as it can get.

I also had a deployment utility in mind that would do something very similar with the yaml, so you wouldn't have to maintain custom deployment scripts for all your contracts every time you worked on something different. The default args are pre-programmed in the yaml (and overrideable on command line). Perhaps that is trying to automate too much, but I definitely think there is a fairly standard workflow here that can be condensed into less overall decision-making and custom code. This all leverages the same contracts.json artifacts file (hopefully standardized) so you can use the tools with any EVM-compatible language with zero changes to the tool.

I had a few other ideas that would end up leveraging this same set of files (a tool for creating botnets for interactive dapp testing, a contract terminal for interacting with your contracts, code coverage and gas profiling from a bytecode => source code mapping, a formal verification engine, etc). All you basically need is the ABI and the bytecode/runtime for any of these tools. There is no need to link in custom handling for different languages, it's all JSON and human-readable YAML config files. You can just focus on writing python code that does what you want instead of writing glue code.

---

Please note Vyper wouldn't need to use either of these config files, all config files will be optional and overridable to new locations. I have been thinking about these ideas for the general dapp workflow.

[carver]

I basically want to let you move very quickly to the testing part

From my perspective, it does the opposite. In order to get started, I have to learn a new config language. Or if it's all generated automatically, then it seems like it could all go into the ethpm lockfile.

It might make sense to get some chat time or face-to-face time to work it out. There's a lot of moving parts. (and definitely with @pipermerriam there, since I only have a surface-level understanding of ethpm).

[fubuloubu]

From my perspective, it does the opposite. In order to get started, I have to learn a new config language. Or if it's all generated automatically, then it seems like it could all go into the ethpm lockfile.

I guess in either scenario, you will have to enter in the default deployment arguments at some point. I prefer entering them in config files versus python test code so I can keep those concerns separate, but I definitely see the argument to have them living in Python. It wouldn't be that much more verbose to write:

package/tests/conftest.py:

@pytest.fixture
def Token(contract_factory):
     return contract_factory('Token', args=MY_ARGS)

I don't think the package is the appropriate place to manage default args however, if that's what you're getting at.

[fubuloubu]

It might make sense to get some chat time or face-to-face time to work it out. There's a lot of moving parts. (and definitely with @pipermerriam there).

I'm definitely on board with discussing if standardizing these "compiler artifacts" could be simply leveraging the ethpm package system for packaging your entire project, and then leveraging that standard in all our other tools. Totally would be willing to talk about that. Are you going to be at ETH Denver? I know piper and @DavidKnott are (as am I)

[fubuloubu]

@carver I edited everything in all 3 comments above a few times, my apologies if you were already done reading it. It's late 👅

[fubuloubu]

Looking more into it, there's definite synergy with ethpm and storing the abi/bin/runtime and perhaps a source code checksum of a contract in an additional contracts listing (alongside sources), or storing that contracts listing at another IPFS location (since it will probably be very large). Then you could avoid recompiling assets when using a package and a host of other things, basically having web3 point to those results and leverage them in the full dapp (avoiding the need to keep track of the ABI ugh).

That's a lot to think about.

[pipermerriam]

Some scattered thoughts:

This library is almost definitely a piece of the puzzle, assuming the ethpm spec is going to be part of this (which I strongly believe it should but i'm particularly impartial when it comes to ethpm)

https://github.com/pipermerriam/py-ethpm

The readme is going to be a bit outdated but the general concept is that it acts as a layer between a standard definition for contract assets (ethpm-spec) and a python interface for interacting with those assets (web3.py). All that's needed assuming ethpm-py was fully written and functional was a way to tell ethpm-py where all the contract assets are located and to load them.

Deployment arguments and what-not are complicated, but I think I agree with @carver that I'd rather see this done with python scripts rather than a custom config format. The other option is to create an EIP with a DSL of some sort that builds on top of ethpm to define a standard for generic scripted deployments.

[fubuloubu]

Hmm, generic scripted deployments. Interesting thought...

Many of the dapps I've been making leverage smart contracts as temporary entities (unlike a token). Their parameters are mostly computed on a case-by-case basis, and I think Web3.{js,py,...} generically meets this deployment requirement today.

My use case for recording the deployment parameters is pretty much strictly for deployment of more permanent contracts (make sure we get the token parameters right) or for testing (so I don't have to assign this every time it's used in a test).

The one thing the config file buys you is that it can be leverage between both use cases, but ultimately I think I am starting to agree with you that has limited utility. Defining them twice in two different python scripts isn't that bad, especially when they have different uses.

---

I definitely think ethpm should be leveraged for any EVM compiler output. It would give us a great standard to build off of, and storing these projects on IPFS would make them more secure when your user loads the dapp from the interwebs (through IPFS basically).

I've been working on the concept of composition for Vyper (instead of inheritance in Solidity), and we were talking about a sort of app store for different pieces of the puzzle (e.g. an Owner type, ERC20, other useful types) to leverage in building larger smart contracts. ethpm would be the perfect datastore for this IMO.

---

@pipermerriam I'll be at ETH Denver, arriving early on the 13th. Want to get together and chat about this along with @DavidKnott and @carver?

[pipermerriam]

I will be here (I live in Boulder which is 20-30 minutes from Denver). Lets meet up.

[carver]

we were talking about a sort of app store for different pieces of the puzzle (e.g. an Owner type, ERC20, other useful types) to leverage in building larger smart contracts. ethpm would be the perfect datastore for this IMO.

Yeah, ethpm absolutely aspires to be a piece of the puzzle that supports a repository of contracts.

---

I won't be at ETH Denver unfortunately. Luckily, Piper and I agree on a whole lot about this topic, so you shouldn't miss too much in my absence. :)

[fubuloubu]

Woot woot consensus

[jacqueswww]

With the latest PR (https://github.com/ethereum/vyper/pull/801) I created, it became clear that having both py-evm and pyethereum on the same project is no longer an option, because of conflicting rlp version required in the two.

I have locked down the py-evm version. But I think my top priority next week will be to move to eth tester.

[fubuloubu]

@jacqueswww use https://github.com/fubuloubu/pytest-ethereum for inspiration!

[jacqueswww]

Definitely! 😎might still opt to use it behind our conftest.py if it provides all necessary functions.

[fubuloubu]

@jacqueswww yeah, the gas estimation thing wasn't a part of my thought process and would be difficult to graft in well. I still think it would be nice to add, perhaps by setting the transaction gas limit to the estimate that Vyper provides (Solidity would use a default)

[pipermerriam]

@jacqueswww please reach out for support/help if/when you want/need it. Happy to help you navigate this change.

[jacqueswww]

@pipermerriam thanks, will do!