eth-clients / holesky

the holesovice post-merge testnet configuration.
Do What The F*ck You Want To Public License
320 stars 103 forks source link

Mass distributions via transactions instead of the genesis state #9

Closed q9f closed 1 year ago

q9f commented 1 year ago
          Maybe it'd be better to do these kind of mass distributions via transactions instead of the genesis state?

The genesis state is baked directly into most execution clients, and I think it'd be much more convenient if it's kept small like the other testnet genesis states :)

I've written a small tool that we're using in the ephemery testnet to do individual fund distributions after each reset. It's designed to work with a large set of addresses, so maybe it's worth to consider it for this too. https://github.com/pk910/testnet-funding-tool

Edit: I've been curious to see how the tool works with such a big address set. It's working quite well. I've distributed 1 ETH to each of the eligible addresses timbeiko posted before on the ephemery testnet. That's a set of 497596 addresses.

It took about 5h to complete (20 tx per block with batches of 20 transfers each, so 400 transfers per block). Could have been completed in 2,5h too by using the full available block size.

Nice side effect: I didn't know that so many of my own dev addresses were eligible for the grabteeth.xyz drop :D I never checked them all one by one, but due to my distribution test I can easily identify them now, as they're all having a balance of 1 ETH on ephemery now

Originally posted by @pk910 in https://github.com/eth-clients/holesky/issues/1#issuecomment-1452860234

pk910 commented 1 year ago

Heya, Great to see that this Idea is being liked :)

I've made some changes to the funding-tool, that allows it to run offline, just generating a list of signed transactions. These signed transactions can then be broadcasted by anyone who likes once the network is live. The tool basically deploys the distributor contract and uses that for all subsequent transactions, due to the increasing tx nonce all transactions are forced to run in order.

The tool can also be run from a gihub action from within this repository. When doing so, the private key could either be random and destroyed afterwards (it's actually no longer needed, as the list of signed transactions is everything this wallet should ever be allowed to do), or it could be read from a github secret (so it's still there in case of an emergency, but outsiders like me don't have access to it).

I think from the previous discussions that we want to start with a "base"-list of eligible wallets? Should we just take the grabteeth.xyz list, or filter it further?

In addition there should be another tracking issue for arbitrary address collection. I guess we can't fully prevent such an address collection issue from being spammed, but maybe we can enforce some "basic" rules.

I've thought about implementing a kind of comment moderation via github actions too: https://github.com/pk910/funding-list

When adding/editing a comment in "https://github.com/pk910/funding-list/issues/1" a github action is triggered that checks if:

There could be more strict checks added easily. eg. a check for a gitcoin passport?

Happy for feedback :) If you like the Idea I can add these action workflows via a PR

sambacha commented 1 year ago

Why not use a pull based contract?

use OAuth2 and see user activity across github

pk910 commented 1 year ago

That's not really gonna work without some gas to do the pull transaction. You'd have to go to a faucet first, just to use the contract for the proper funding. That's not really user friendly compared to just dropping some coins into their wallet :)

Second problem I see is visibility. With a pull based contract most of the eligible users won't even know they're eligible for the drop... I was quite surprised to see how many of my own addresses were eligible for the grabteet drop. I guess most users didn't ever check any of their addresses as they just don't know about it.

pk910 commented 1 year ago

I've opened PR #10, which adds a workflow to run the funding tool from within this repository. That way it's fully trust-less and does only rely on someone (probably @q9f) triggering this workflow. The output from the workflow (signed list of transactions) can be submitted by anyone once the network is live.

We should further discuss which addresses should be eligible. With the big grabteeth list of 497,596 addresses that's a total of 4,975,960,000 ETH if we distribute 10,000 ETH to each. That sounds quite high.. Don't know if that's desired to have so much funds in the network? (I'm slightly worried about all type of chain spamming)

pk910 commented 1 year ago

I've composed another list of addresses, which contains addresses that successfully deposited to the beaconchain on goerli or mainnet. As the network is more intended for CL testing, I think it's more reasonable to pre-fund these addresses instead of wallets that deployed a contract.

My final list looks as follows: Goerlli [txt / csv] 11350 Addresses Mainnet: [txt / csv] 20680 Addresses Combined: [txt] 31773 Addresses

To compose the lists I've done the following steps:

  1. I've crawled all Deposit Events from the deposit contracts and loaded the corresponding transaction details via this script The raw output of the script can be found here: goerli / mainnet It includes the validator pubkey, withdrawal address if set via 0x01 creds on deposit, block height, depositor address & forwarder contract if sent via another contract.
  2. I've joined the current validators state via a json dump from the getState api. Invalid deposits were removed during this merge operation as there is no corresponding validator state for them.
  3. Grouped the data by the latest withdrawal credentials. This basically combined all validators that use the same withdrawal address to a single entry in the list. I've preserved the first depositor address for each group for the next step.
  4. Grouped the data by the depositor address. This basically combined all validator groups from the last step that were deposited from the same source wallet.
  5. Filtered out well known entities like Lido and CEX pools like Kraken/Coinbase/...

Steps 2-5 are done by a excel sheet, which unfortunately got too big for github (>100MB each), I can share it if anyone is interested in the aggregation steps in particular :)

However, I'm also open for other aggregations if anyone has a better way to do it. If anyone wants to play around with the data, I've added a link to the raw outputs of the crawler above.

In the last holesky planning call it has also been decided to keep the amount of funds in the network limited to "a few 100M" and the mass distribution should be lowered to ~1-2k for each address.

So, there are various ways to go forward. Some examples:

  1. We use the grab-teeth list of 497k addresses to distribute 1k HolETH to each. Total Amount: 497M HolETH (pretty much already exceeds the intention of a few 100M supply cap)
  2. We use the combined list of depositor addresses (31k addresses) to distribute 2k HolETH to each. (enough for 62 validators) Total Amount: 31M HolETH
  3. We use a combination of both lists, to distribute 100 HolETH to all addresses from the grab-teeth list & 1k HolETH to the list of goerli/mainnet depositors. Total Amount: ~65M HolETH

Some input would be great as I don't really want to decide that alone :)

mcdee commented 1 year ago

I'm wondering if there is a significant benefit to doing this compared to having an address with 50M ETH fronted by a faucet, where people can fetch 33ETH at a time with suitable restrictions.

Note that we can always create a second address with 50M ETH, which can be used to distribute funds via the above method and one of the lists at a later date. As long as we have the ETH lying around, we don't have to be in a rush to distribute it.

sambacha commented 1 year ago

Why not just use SELFDESTRUCT to distribute all the ETH? It should be the most scaleable way to do so in fact.

pk910 commented 1 year ago

@mcdee The reason why I am pushing this is, that we have the ability to do this mass distribution fully trust-less by publishing a signed list of transactions before genesis. These transactions can then be checked and the origin wallet funded via the genesis state. After the network is running, these transactions can be broadcasted by whoever wants to do so (I will do so, but it's not forced to be me). I agree we can do this mass distribution after genesis, but that's no longer trust-less then, as it then depends on someone doing this at some time.

I'd suggest adding some "reserve wallets" controlled by EF anyway. If these reserve funds are never used, fine. But we should absolutely avoid running into the same situation as goerli again. It's ultra nerve-wracking for so many devs...

@sambacha Can you explain more? I've tried the SELFDESTRUCT way, but always ended up paying more fees than with regular calls.

sambacha commented 1 year ago

@mcdee The reason why I am pushing this is, that we have the ability to do this mass distribution fully trust-less by publishing a signed list of transactions before genesis. These transactions can then be checked and the origin wallet funded via the genesis state. After the network is running, these transactions can be broadcasted by whoever wants to do so (I will do so, but it's not forced to be me). I agree we can do this mass distribution after genesis, but that's no longer trust-less then, as it then depends on someone doing this at some time.

I'd suggest adding some "reserve wallets" controlled by EF anyway. If these reserve funds are never used, fine. But we should absolutely avoid running into the same situation as goerli again. It's ultra nerve-wracking for so many devs...

@sambacha Can you explain more? I've tried the SELFDESTRUCT way, but always ended up paying more fees than with regular calls.

https://gist.github.com/sambacha/25e167ef0a5093f206bb5a9ba9900b57

q9f commented 1 year ago

I think one beneficial side effect of doing this is having a more complex state trie after genesis, and this was always a desired feature for new testnets. I don't really see much of a downside here.

sambacha commented 1 year ago

If you did not see my last comment here is the contract:



pragma solidity >=0.7.6 <0.9.0;

contract TransactionConduit {

    function massTeleport(address payable[] memory _recipients, uint[] memory _amounts) public payable returns (bool) {

      uint numRecipients = _recipients.length;

      require(numRecipients == _amounts.length, 'FAIL! PARAM SIZE');

Conduit unstablePortal = new Conduit(); 

      for (uint i=0; i < numRecipients; i++) {
        unstablePortal.teleportEtherTo{value: _amounts[i]}(_recipients[i]);
      }

      uint remainingBalance = address(this).balance;

      if (remainingBalance > 0) {
        unstablePortal.teleportEtherTo{value: remainingBalance}(payable(msg.sender));
      }

      return true;
}

    function voidEther() public payable {
      Conduit unstablePortal = new Conduit();
      unstablePortal.teleportEtherTo{value: msg.value}(payable(address(unstablePortal)));

    }
}

contract Conduit {
  function teleportEtherTo(address payable _recipient) public payable {
    selfdestruct(_recipient);
  }
}
pk910 commented 1 year ago

@sambacha I've seen it and I tried modifying my distribution tool to use it. Unfortunately I've always ended up paying more fees than with normal transfers.

Example with 20 transfers to new wallets (no state yet): Normal transfers: 725,991 Gas tx SELFDESTRUCT transfers: 905,562 Gas tx With exactly your contract code: 909,729 Gas tx

Am I missing something?

zhongfu commented 1 year ago

@pk910 if I'm understanding this correctly, the benefit is that it allows you to send Ether to smart contract recipients without executing any of their code

but since this is to be done at genesis, I'm not too sure that's going to be an issue to begin with?

maybe I'm missing something here as well

pk910 commented 1 year ago

Aah, fair point 👍 and totally agree that it is a reasonable thing to consider in some situations.

However, I think for this initial fund distribution it's not necessary, because there are no contracts deployed at genesis. And because our funding list contains non-contract addresses that deployed some contracts before only, there can't be any code deployed on any of these addresses itself.

q9f commented 1 year ago

@pk910 do we expect issues wrt nonces?

pk910 commented 1 year ago

Not that I'm aware of. The nonces actually ensure that all transactions are executed in strict order and no funds are sent to the distributor contract before deployment or so. If any of the transactions gets dropped from the mempool it can simply be re-broadcasted.

q9f commented 1 year ago

Done, thanks for this. :)

https://github.com/eth-clients/holesky/actions/runs/5991155800

sealer3 commented 1 year ago

It looks like only 491 of the 11,350 addresses in @pk910's goerli depositor list are part of funding-list.txt. Is only the grabteeth list being used? I'd imagine a lot of current goerli stakers would be happy to roll onto holesky if their addresses are funded. Moreso to support them in testing staking than because we would need them.

pk910 commented 1 year ago

Yea, that's correct. I actually didn't receive any feedback on the second list I've composed, so as time was going on we've used the original grabteeth list only.

We cannot change the list anymore at this point in time (transactions are signed and published - no way to undo that). However, we could probably organize a second smaller drop for the addresses on my second list.

There is an assignment of 500,000,000 ETH for the distribution. 497,596,000 ETH will be distributed, some fees will be burned (<100ETH), so there are about 2,403,900 ETH left.

If we take the combined list of 31,773 addresses for mainnet/goerli depositors, we could distribute about 75 ETH to each. I could top it up to 100ETH by using some of my faucet assignment too.

Still not that much, but still enough to spin up 3 validators :)

@q9f What do you think about it? You're the key holder ;)

q9f commented 1 year ago

We can top it to 100 sure. If we had known this a couple of days earlier, we could have bumped the initial balance so that we have a bigger buffer.

pk910 commented 1 year ago

If we had known this a couple of days earlier, we could have bumped the initial balance so that we have a bigger buffer.

Yea, completely agree. And I'm slightly sorry for that.

However, from my point of view, I'm just a regular holesky user, no special client dev role or similar assignment in the ETH world (yet). So, if I propose a completely separate list of addresses to fund, I'd expect at least a small feedback like "yea, let's do it" or so from a relevant person. Otherwise I see it as not accepted and continue with the original proposal.

Maybe I should just see such updated proposals as accepted in future if nobody complains.. Or ask for feedback more "aggressively"?

Anyway, that's how it is now and I think it's not bad at all to fund contract deployers. For distribution of the remaining balance to depositors we should wait for the initial distribution to run through. I don't want to change the signed transactions at that point in time. I think it's best to do it after genesis

q9f commented 1 year ago

watch it getting drained 🤓

https://holesky.etherscan.io/address/0xd3994e4d3202dd23c8497d7f75bf1647d1da1bb1