search

erigontech / erigon

Ethereum implementation on the efficiency frontier https://erigon.gitbook.io
GNU Lesser General Public License v3.0
3.15k stars 1.13k forks source link

Unable to find any doc to run local devnet using Erigon #2588

Closed Uttam-Singhh closed 3 years ago

Uttam-Singhh commented 3 years ago

I want to run Erigon on my local devnet but I am unable to find any documents or any steps to do so?? I am a beginner please let me know if there is any document or article.

Here is an example of running local devnet using Geth - https://medium.com/datawallet-blog/how-to-deploy-a-local-private-ethereum-blockchain-c2b497f068f4 I really want any article like this for Erigon too.

Also, the discord link to join the server is expired, Please send me another invite on - uttamkhanduja@yahoo.in if possible.

I really want to explore erigon!

Uttam-Singhh commented 3 years ago

@AskAlexSharov I will grateful if you can help!

AskAlexSharov commented 3 years ago

Article is kind of valid, but we don't have cli command to set genesis (probably now only hardcode new genesis can help - you can search ErigonChainConfig in source code and use that variable as an example), we don't have admin.AddPeer api (probably can use --staticpeers cli flag), and maybe we don't have something else.

Not sure that we are have beginners-friendly way of create dev-net. Maybe in future.

mmgen commented 3 years ago

Geth has this functionality, but it was removed in Erigon for some reason. I would like to see it restored. For wallet devs, a working devnet is essential.

Uttam-Singhh commented 3 years ago

So there is no other way to start a stand-alone chain?? @AskAlexSharov @mmgen

mmgen commented 3 years ago

So there is no other way to start a stand-alone chain?? @AskAlexSharov @mmgen

As far as I know, no.

Uttam-Singhh commented 3 years ago

@AskAlexSharov I'd love to work on this functionality to init a custom chain with its own genesis or bring back the functionality if it was already there! Should I create a new issue?? and also I would be really grateful if someone can guide me more on this!

AskAlexSharov commented 3 years ago

Sure.

I advise you start from re-introducing geth init to setup custom genesis block. We have general ticket: https://github.com/ledgerwatch/erigon/issues/2030 (if you don't need geth import it's fine). You can get source code of geth import - I expect that amount of required changes will not be very big. Feel free to do it in multiple PR's - do as comfortable or as make sense.

We have only 1 non-obvious requirement:

You can ask questions in Discord, me or somebody online will answer.

I will start from another side - I will check is it easy to mine blocks for dev-net or not. Do you need dev-net based on Ethash or Clique is also fine?

Uttam-Singhh commented 3 years ago

Sure.

I advise you start from re-introducing geth init to setup custom genesis block. We have general ticket: #2030 (if you don't need geth import it's fine). You can get source code of geth import - I expect that amount of required changes will not be very big. Feel free to do it in multiple PR's - do as comfortable or as make sense.

We have only 1 non-obvious requirement:

  • we don't wan't have Account Manager - don't want store private key in Erigon's memory or generate them. Because we believe it will simplify for us Security Audit in future.

You can ask questions in Discord, me or somebody online will answer.

I will start from another side - I will check is it easy to mine blocks for dev-net or not. Do you need dev-net based on Ethash or Clique is also fine?

Thanks for your reply!

I don't need geth import. I just want to run a stand alone chain(local devnet).

Also as you advised, how can I start re-introducing geth init, what are the changes that has to done? Do I need to do changes in Config.go file under params ? please provide a reference if possible!

mmgen commented 3 years ago

We have only 1 non-obvious requirement:

  • we don't wan't have Account Manager - don't want store private key in Erigon's memory or generate them. Because we believe it will simplify for us Security Audit in future.

Parity has a hard-coded faucet address with associated private key for devnet. Would such an approach be acceptable?

Do you need dev-net based on Ethash or Clique is also fine?

I can't speak for @Uttam-Singhh, but the Clique engine (mines blocks automatically whenever a tx appears in the mempool) would be fine for me. I'm not interested in mining.

Uttam-Singhh commented 3 years ago

Do you need dev-net based on Ethash or Clique is also fine?

I can't speak for @Uttam-Singhh, but the Clique engine (mines blocks automatically whenever a tx appears in the mempool) would be fine for me. I'm not interested in mining.

Yeah Clique works for me too!

AskAlexSharov commented 3 years ago

what are the changes that has to done? - I don't know - but I'm sure that part of code is not supper different from geth.

Uttam-Singhh commented 3 years ago

what are the changes that has to done? - I don't know - but I'm sure that part of code is not supper different from geth.

oh okay!😶 @mmgen Do you have any Idea??

mmgen commented 3 years ago

@mmgen Do you have any Idea??

I'm not that familiar with the code, nor am I a Go programmer, but for starters you might look at DeveloperGenesisBlock() in core/genesis.go.

AskAlexSharov commented 3 years ago

About next steps in article https://medium.com/datawallet-blog/how-to-deploy-a-local-private-ethereum-blockchain-c2b497f068f4

  1. i think you don't need geth account new command: create private/public key pair by any other utils. For example: https://kobl.one/blog/create-full-ethereum-keypair-and-address/ Or by geth. Or by Metamask. Or any other way. Erigon has code to create address: 

    privateKeyECDSA :=ecdsa.GenerateKey(S256(), rand.Reader)
address := crypto.PubkeyToAddress(privateKeyECDSA.PublicKey)

    but we will not add command to generate private keys for users - by security reasons.

  2. When Erigon node starts it prints enode id: Started P2P networking version=65 self=enode://7d461b20dda15886c17bf30af7bf101ad94db0567d87c9a357bc159a3c765d4f54435fe48e1d695518e7e41068c7dcb1f274c5bc594ad3c0b0b8c48ce3de078e@127.0.0.1:30304 name=erigon/v2021.09.2/darwin-amd64/go1.17 collect all of them and pass to --staticpeers cli flag If you like - you can copy from geth functionality which allow storing static peers in .toml file

  3. Only way to send transactions by Erigon is to use RPC method eth_sendRawTransaction You will need find some way to create raw transaction.

  4. We have cli flags to start miner and set etherbase.

So, just go ahead step by step.

Uttam-Singhh commented 3 years ago

Thank you @AskAlexSharov for guidance!

  1. i think you don't need geth account new command: create private/public key pair by any other utils. For example: https://kobl.one/blog/create-full-ethereum-keypair-and-address/ Or by geth. Or by Metamask. Or any other way.

I will use Metamask for this but what about nodes? I will have to create local nodes or not?

  1. When Erigon node starts it prints enode id: Started P2P networking version=65 self=enode://7d461b20dda15886c17bf30af7bf101ad94db0567d87c9a357bc159a3c765d4f54435fe48e1d695518e7e41068c7dcb1f274c5bc594ad3c0b0b8c48ce3de078e@127.0.0.1:30304 name=erigon/v2021.09.2/darwin-amd64/go1.17 collect all of them and pass to --staticpeers cli flag

Are you talking about running the main chain? I need to collect all of them from the running main chain or any other network?

AskAlexSharov commented 3 years ago

“ I will have to create local nodes or not?” - yes, same as in article.

“ Are you talking about running the main chain?” - no, you will create some nodes for your chain.

mmgen commented 3 years ago

but we will not add command to generate private keys for users - by security reasons.

How about using a hard-coded private key/address for the faucet, as OpenEthereum does? Then you could just start up in devmode with a single command-line flag, say --chain=dev, with no setup required.

AskAlexSharov commented 3 years ago

Ah, yes, need to add

Uttam-Singhh commented 3 years ago

@AskAlexSharov I am stuck please help! I am able to run the devnet, i also connected it using metamask, I tried to do a transaction it showed the transaction was submitted 👇

Screenshot 2021-09-07 at 12 51 40 PM

But the blocks are still not increasing, they are stuck...🤷‍♂️

AskAlexSharov commented 3 years ago

Show me logs of Erigon

Uttam-Singhh commented 3 years ago

Here it is - The transaction was fired at 12.48.30

Screenshot 2021-09-07 at 1 25 48 PM
AskAlexSharov commented 3 years ago

And cli command to run erigon?

Uttam-Singhh commented 3 years ago

./build/bin/erigon --datadir dev --chain dev --mine --miner.sigkey=

AskAlexSharov commented 3 years ago

I think --miner.sigkey flag was introduced by mistake. Let me take a look.

Uttam-Singhh commented 3 years ago

Ok! Please guide me further once you have a look!

AskAlexSharov commented 3 years ago

Added hard-coded private key for --chain dev https://github.com/ledgerwatch/erigon/pull/2639 you can remove --miner.sigkey flag

0xKrishna commented 3 years ago

Gives this error now


krishna@ ~/erigon (devnet_private_key) $ erigon --datadir dev --chain dev --mine
INFO[09-07|14:31:37.967] Build info                               git_branch=krishna/add-init git_tag=v2021.08.01-105-g2e9718599 git_commit=2e9718599556d7280bd3a109c7de061adeea9cf6
INFO[09-07|14:31:37.967] Starting Erigon in ephemeral dev mode...
INFO[09-07|14:31:37.971] Maximum peer count                       ETH=100 total=100
INFO[09-07|14:31:37.971] Set global gas cap                       cap=50000000
Fatal: Please specify developer account address using --miner.etherbase```
0xKrishna commented 3 years ago

When miner.etherbase is used

krishna@ ~/erigon (devnet_private_key) $ erigon --datadir dev --chain dev --mine --miner.etherbase 0x03f50EE89696c10Ec85Db312535D52fF5868b29A
INFO[09-07|14:32:22.599] Build info                               git_branch=krishna/add-init git_tag=v2021.08.01-105-g2e9718599 git_commit=2e9718599556d7280bd3a109c7de061adeea9cf6
INFO[09-07|14:32:22.599] Starting Erigon in ephemeral dev mode...
INFO[09-07|14:32:22.601] Maximum peer count                       ETH=100 total=100
INFO[09-07|14:32:22.602] Set global gas cap                       cap=50000000
INFO[09-07|14:32:22.602] Using developer account                  address=0x03f50EE89696c10Ec85Db312535D52fF5868b29A
INFO[09-07|14:32:22.603] Opening Database                         label=chaindata path=dev/chaindata
INFO[09-07|14:32:22.614] database closed                          label=chaindata
INFO[09-07|14:32:22.614] Opening Database                         label=chaindata path=/Users/krishna/erigon/dev/chaindata
INFO[09-07|14:32:22.628] Initialised chain configuration          config="{ChainID: 1337 Homestead: 0 DAO: <nil> DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul: 0, Muir Glacier: 0, Berlin: 0, London: <nil>, Engine: clique}"
INFO[09-07|14:32:22.636] Initialising Ethereum protocol           network=1337
INFO[09-07|14:32:22.637] Effective                                prune="--prune="
INFO[09-07|14:32:22.637] Starting private RPC server              on=127.0.0.1:9090
WARN[09-07|14:32:22.638] Preverified hashes not found for         chain=
INFO[09-07|14:32:22.642] Transaction pool price threshold updated price=0x1
EROR[09-07|14:32:22.642] Etherbase account unavailable locally    err=nil
EROR[09-07|14:32:22.642] Erigon startup                           err="signer missing: <nil>"
AskAlexSharov commented 3 years ago

@0xKrishna added default sign key for Devnet: https://github.com/ledgerwatch/erigon/pull/2639 Now must be enough: --chain dev --mine

AskAlexSharov commented 3 years ago

@0xKrishna are you on https://github.com/ledgerwatch/erigon/pull/2639 ?

0xKrishna commented 3 years ago

Sorry, My bad

krishna@ ~/erigon (devnet_private_key) $ ./build/bin/erigon --datadir dev --chain dev --mine

Working now

AskAlexSharov commented 3 years ago

To continue: @Uttam-Singhh you can see in Erigon logs "local tx propagated" - it means your transaction came to TxPool successfuly and TxPool sent it to other peers by P2P. Now need understand if your transaction came to mined block. Mining happening next way:

Uttam-Singhh commented 3 years ago

Oh okay I will try it out! Thank you so much @AskAlexSharov!❤️

AskAlexSharov commented 3 years ago

after https://github.com/ledgerwatch/erigon/pull/2639 you better delete your dev db and of course now - if you do custom genesis - you need allocate some funds for default etherbase

0xKrishna commented 3 years ago

I have tried everything but my node is not mining blocks at all, Interesting part is this

INFO[09-07|17:03:45.136] [1/18 Headers] Wrote block headers       number=0 blk/second=0.000 alloc="10.51 MiB" sys="72.14 MiB"
^CINFO[09-07|17:04:12.572] Got interrupt, shutting down...
INFO[09-07|17:04:12.574] event subscription channel closed with the RPC daemon
INFO[09-07|17:04:12.577] [1/5 MiningCreateBlock] Start mine       block=1
INFO[09-07|17:04:13.579] Transaction pool stopped

When I try to stop the node by interrupting, I can see a log related to mining create block

0xKrishna commented 3 years ago

Using similar command in geth works fine

 ./build/bin/geth --datadir dev --dev --mine

INFO [09-07|18:59:44.165] Commit new mining work                   number=1 sealhash=e891d0..d0aa79 uncles=0 txs=0 gas=0 fees=0 elapsed="143.589µs"
INFO [09-07|18:59:44.165] Sealing paused, waiting for transactions

But in erigon it doesn't mine

./build/bin/erigon --datadir dev --chain dev --mine

INFO[09-07|19:19:47.207] [1/18 Headers] Wrote block headers       number=0 blk/second=0.000 alloc="7.33 MiB" sys="72.08 MiB"
INFO[09-07|19:20:17.179] [1/18 Headers] Wrote block headers       number=0 blk/second=0.000 alloc="7.17 MiB" sys="72.08 MiB"
mmgen commented 3 years ago

Using similar command in geth works fine


 ./build/bin/geth --datadir dev --dev --mine

Is the --mine option really required? For Geth devnet, all I need is the following, nothing more:

geth --http --http.api=eth,web3,txpool --dev

Sealing paused, waiting for transactions is coming from the Clique engine, which is what you want. Maybe Clique is not getting activated with Erigon?

AskAlexSharov commented 3 years ago

I found problem. Mining was blocked on some mutex which didn't happen on active networks(where periodically arriving blocks). Here is some fix: https://github.com/ledgerwatch/erigon/pull/2642

0xKrishna commented 3 years ago

Okay, This is what's happening now

INFO[09-08|11:39:41.333] local tx propagated                      to_peers_amount=0 tx_hash=0xed4eb1564a7d09d04a11fd4998924b1d3b1ddd9a70bd339904c7202b739eddb9
INFO[09-08|11:39:42.908] [1/18 Headers] Processed                 highest inserted=0 age=52y5mo1w
INFO[09-08|11:39:42.909] Commit cycle                             in=40.499µs
INFO[09-08|11:39:42.909] Update current block for the RPC API     from=1 to=0
INFO[09-08|11:39:42.909] [1/5 MiningCreateBlock] Start mine       block=1
INFO[09-08|11:39:42.910] [4/5 IntermediateHashes] Regeneration trie hashes started
INFO[09-08|11:39:42.910] [4/5 IntermediateHashes] Trie root       hash=0xe3eade507848eaf22077585b83e58bd8051d584ef9057c3423ec9aab1a0b7cc8
INFO[09-08|11:39:42.910] [4/5 IntermediateHashes] Regeneration ended
INFO[09-08|11:39:42.910] [5/5 MiningFinish] start
INFO[09-08|11:39:42.910] [5/5 MiningFinish] block ready for seal  blocn_num=1 transactions=2 gas_used=42000 gas_limit=11488771 difficulty=2
INFO[09-08|11:39:42.911] [1/18 Headers] Waiting for headers...    from=0
INFO[09-08|11:39:44.908] [1/18 Headers] Processed                 highest inserted=0 age=52y5mo1w
INFO[09-08|11:39:44.910] Commit cycle                             in=33.766µs
INFO[09-08|11:39:44.910] Update current block for the RPC API     from=1 to=0
INFO[09-08|11:39:44.910] [1/18 Headers] Waiting for headers...    from=0

Block is ready for seal but the number is still not increasing, Anything else needs tp be done? Command used to run the chain

./build/bin/erigon --datadir dev --chain dev --mine
AskAlexSharov commented 3 years ago

but the number is still not increasing, Anything else needs tp be done? I think easiest way for now is to start multiple nodes and join them by --staticpeers flag. They will propagate mined block to each-other.
I will work to apply own mined blocks tomorrow - but I'm not sure - is it easy or hard.

0xKrishna commented 3 years ago

I used --staticpeers but how can I make sure they are connected? Running both the nodes on the same machine by changing the port number from 9090 to 9091. That's fine, Right? I used RPC of one node to submit the txn, It had all the earlier logs but there were no logs on the 2nd node, Nor the txn got propagated on 2nd node.

AskAlexSharov commented 3 years ago

9090 - it's grpc port. you need flag --port 30303

AskAlexSharov commented 3 years ago

https://github.com/ledgerwatch/erigon#default-ports-and-protocols--firewalls

0xKrishna commented 3 years ago

Yes, I used this command to run 2nd node

 ./build/bin/erigon --datadir dev2 --chain dev --mine --staticpeers "enode://5c754c68a5f1d489c1273c2fc5c4926194e7f6588a90733b74f6f8292aa20a6585b2d22ae112d65d3f5407d0302843e8c3d174970e6d5ebe2b329532bb8f61a5@127.0.0.1:30304?discport=0" --private.api.addr=localhost:9091 --port 30305 --p2p.eth65.port 30306

1st node is using ports 9090, 30303 and 30304 I hope staticpeers is used correctly.

AskAlexSharov commented 3 years ago

Looks correct

0xKrishna commented 3 years ago

I am getting this in logs

INFO[09-08|18:13:29.290] [p2p] GoodPeers                          eth66=0 eth65=0
AskAlexSharov commented 3 years ago

Apply mined blocks: https://github.com/ledgerwatch/erigon/pull/2650

0xKrishna commented 3 years ago

Thanks @AskAlexSharov , It's mining blocks now, Header number moving ahead with transactions.

0xKrishna commented 3 years ago

Single node is working fine now and mining blocks as per the transaction, But if I run another local node and also use --staticpeer flag, The 2nd node is running independently and not syncing with the first one. Any suggestion?

mandrigin commented 3 years ago

@0xKrishna can you give here both full command lines here?