search

protolambda / eth2-val-tools

Some experimental tools to manage validators - use at your own risk
MIT License
63 stars 21 forks source link

keystore: paralellize key generation #5

Closed skylenet closed 2 years ago

skylenet commented 2 years ago

This got to my attention when @parithosh was generating a lot of keys for testnet purposes. It was taking him a long time.

This change will benefit from all your CPUs and paralellize the creation of the keystores.

Sample timings on my local machine (4 cores with hyperthreading) when creating 1000 keys from a mnemonic:

$ time eth2-val-tools-old keystores --source-mnemonic="$MNEMONIC" --source-min=0 --source-max=1000 --prysm-pass lol --out-loc test1
# 285.51s user 1.52s system 101% cpu 4:43.79 total

$ time ./eth2-val-tools keystores --source-mnemonic="$MNEMONIC" --source-min=0 --source-max=1000 --prysm-pass lol --out-loc test2
# 620.00s user 1.00s system 752% cpu 1:22.48 total

As you can see, before it took 4min 43sec and on this branch it took 1min 22sec.

skylenet commented 2 years ago

@protolambda I've noticed that you've added the --insecure flag which make it even go faster, that's cool! I've updated the PR with your new changes and solved the code conflicts. I ran the following test locally on my machine:

time ./eth2-val-tools      keystores --source-mnemonic="$MNEMONIC" --source-min=0 --source-max=100000 --prysm-pass Publ1cS3cret --out-loc test_new --insecure
# 666.36s user 42.88s system 715% cpu 1:39.15 total
time ./eth2-val-tools-prev keystores --source-mnemonic="$MNEMONIC" --source-min=0 --source-max=100000 --prysm-pass Publ1cS3cret --out-loc test_old --insecure
# 390.66s user 16.93s system 126% cpu 5:21.41 total

# verfiy pub keys are ordered the same
diff test_old/pubkeys.json test_new/pubkeys.json
# returns nothing, so we're good

Generating 100.000 insecure keys took 1min 39sec compared to previously 5min 21sec