Open ricmoo opened 6 years ago
Sounds great, I'm on it. I'll do a pull request once complete to let you review.
Cool. Could you maybe add a chunk
command to the bin/ethers-airdrop.js as well, which will take in a normal airdrop-balances.json and spit out the chunked version? It's a lot easier to build a file similar to how the example does. I will provide some example files to play with. :)
For sure. Will include that in my update
Quick update so you know where everything is at. I've forked and added the basic additions here: https://github.com/NoahMarconi/ethers-airdrop
Going to work through some test cases and then add the command line chunk
before I create a pull request.
This is a quick idea to break up the file, which will greatly increase the speed. At 1 million records it takes 20-30 minutes to completely collapse the merkle tree, necessary for both computing the merkle root and to compute any given single merkle proof; multiple merkle proofs could be solved at the same time, but the space requirements would be larger.
We can trivially break up a Merkle tree though; we will demonstrate with a smaller tree, with k = 16.
Example 1: Chunk size C = 2 (8 chunk files, 1 root file)
Example 2: Chunk size C = 4 (4 chunks files, 1 root file)
For any given index, the file
floor(index / C)
and the root file are sufficient to reconstruct any merkle proof, the root is sufficient to compute the merkle root. C must be a power of 2.For now we will support C = 1024, which in the case of 1 million records means each chunk is around 65kb and the root file is around 65kb, for a total of 1001 files.