vyperlang / titanoboa

a vyper interpreter
https://titanoboa.readthedocs.io
Other
242 stars 41 forks source link

improvement: fixtures powered profiling #18

Open bout3fiddy opened 1 year ago

bout3fiddy commented 1 year ago

Since @charles-cooper introduced boa's shiny new gas profiler, it brings us closer to a framework that gives vyper devs very granular insights into their contracts. Just as a refresher, one can generate a line-by-line gas profile of their vyper code using the following steps (example):

In [1]: import boa
In [2]: boa.env.enable_gas_profiling()
In [3]: t = boa.load("examples/ERC20.vy", "Token", "TKN", 18, 0)
In [4]: alice = boa.env.generate_address()
In [5]: t.mint(alice, 100)
In [6]: t.line_profile().summary()
Out[6]: 
0x0000000000000000000000000000000000000066/examples/ERC20.vy net_tot_gas: 22215  self.balanceOf[_to] += _value
0x0000000000000000000000000000000000000066/examples/ERC20.vy net_tot_gas: 20144  self.totalSupply += _value
0x0000000000000000000000000000000000000066/examples/ERC20.vy net_tot_gas: 1807   log Transfer(empty(address), _to, _value)
0x0000000000000000000000000000000000000066/examples/ERC20.vy net_tot_gas: 121    assert msg.sender == self.minter  # rekt: non-minter tried to mint
0x0000000000000000000000000000000000000066/examples/ERC20.vy net_tot_gas: 22     assert _to != empty(address)
0x0000000000000000000000000000000000000066/examples/ERC20.vy net_tot_gas: 15     def mint(_to: address, _value: uint256):

One can imagine this can be combined with hypothesis and pytest fixtures to generate line-by-line gas reports. The user would set up a profile_* python method in the following manner (similar to how test_* works for pytest):

from hypothesis import given, strategies, settings

@given(strategies.integers(10**15, (10**10)**3 * 10**18))
@settings(max_examples=10000)
def profile_mint(erc20mock, alice, value):
    with boa.env.anchor():
        erc20mock.mint(alice, val)

To run profiling, it would be similar to pytest: python -m boa-profile -k "profile_mint" And the output would be similar to a line profile, but perhaps with more information (some idea I thought about just now, so maybe not the prettiest way a gas report would look, but I could add more details here, I guess):

line:  self.balanceOf[_to] += _value ; net_tot_gas: 22215 ; std_tot_gas: 2000
line:  self.totalSupply += _value ; net_tot_gas: 20144  ; std_tot_gas: 100
line:  log Transfer(empty(address), _to, _value) ; net_tot_gas: 1807 ; std_tot_gas: 1000
line:  assert msg.sender == self.minter  # rekt: non-minter tried to mint ; net_tot_gas: 121 ; std_tot_gas: 10
line:  assert _to != empty(address) ; net_tot_gas: 22 ; std_tot_gas: 1
line:  def mint(_to: address, _value: uint256): ; net_tot_gas: 15 ; std_tot_gas: 15

The goal would be to save gas reports from this fuzz-style profiling. Devs could commit these reports into their git repositories as well. Gas reports like this could be pretty cool and advantageous for vyper devs.

What's unclear is how this gas report would look / what more detail it would have. Ideas here are welcome!

DanielSchiavini commented 2 months ago

@bout3fiddy is this still necessary? I think it can be easily done already with pytest, I'm not sure if boa needs to change.