search

0xPolygonHermez / cdk-erigon

Ethereum implementation on the efficiency frontier
GNU Lesser General Public License v3.0
28 stars 23 forks source link

Unable to sync: Error downloading blocks from datastream error -- broken pipe #706

Closed northwestnodes-eric closed 1 day ago

northwestnodes-eric commented 6 days ago

System information

Erigon version: hermeznetwork/cdk-erigon:v1.1.5.2-amd64

OS & Version: Ubuntu Linux 22.04

Commit hash:

Erigon Command (with flags/config):

    command:
      - --chain=hermez-mainnet
      - --externalcl=true
      - --maxpeers=32
      - --downloader.disable.ipv6=true
      - --nat=extip:<EXT IP>
      - --zkevm.l2-chain-id=1101
      - --zkevm.l2-sequencer-rpc-url=https://zkevm-rpc.com
      - --zkevm.l2-datastreamer-url=stream.zkevm-rpc.com:6900
      - --zkevm.address-sequencer="0x148Ee7dAF16574cD020aFa34CC658f8F3fbd2800"
      - --zkevm.address-zkevm="0x519E42c24163192Dca44CD3fBDCEBF6be9130987"
      - --zkevm.address-admin="0x242daE44F5d8fb54B198D03a94dA45B5a4413e21"
      - --zkevm.address-rollup="0x5132A183E9F3CB7C848b0AAC5Ae0c4f0491B7aB2"
      - --zkevm.address-ger-manager="0x580bda1e7A0CFAe92Fa7F6c20A3794F169CE3CFb"
      - --zkevm.l1-chain-id=1
      - --zkevm.l1-rpc-url=<L1 RPC URL>
      - --zkevm.l1-rollup-id=1
      - --zkevm.l1-block-range=20000
      - --zkevm.l1-query-delay=6000
      - --zkevm.l1-first-block=16896700
      - --zkevm.rpc-ratelimit=0
      - --zkevm.datastream-version=2
      - --http=true
      - --http.api=eth,debug,net,trace,web3,erigon,zkevm
      - --http.addr=0.0.0.0
      - --http.port=8545
      - --http.vhosts=any
      - --http.corsdomain=any
      - --ws=true

Concensus Layer:

Concensus Layer Command (with flags/config):

Chain/Network: Mainnet

Expected behaviour

Sync.

Actual behaviour

No sync.

Steps to reproduce the behaviour

Run erigon with the aforementioned config flags.

Backtrace

erigon-rpc  | [INFO] [07-04|10:28:49.783] [1/16 L1Syncer] Starting L1 sync stage
erigon-rpc  | [INFO] [07-04|10:28:49.783] [1/16 L1Syncer] No new L1 blocks to sync
erigon-rpc  | [INFO] [07-04|10:28:49.783] [1/16 L1Syncer] Finished L1 sync stage
erigon-rpc  | [INFO] [07-04|10:28:49.783] [2/16 L1InfoTree] Starting L1 Info Tree stage
erigon-rpc  | [INFO] [07-04|10:28:49.783] [2/16 L1InfoTree] Info tree updates      count=0
erigon-rpc  | [INFO] [07-04|10:28:49.783] [2/16 L1InfoTree] Finished L1 Info Tree stage
erigon-rpc  | [INFO] [07-04|10:28:49.783] [3/16 Batches] Starting batches stage
erigon-rpc  | [INFO] [07-04|10:28:49.783] [3/16 Batches] Starting stream           startBlock=0
erigon-rpc  | [INFO] [07-04|10:28:49.783] [3/16 Batches] Reading blocks from the datastream.
erigon-rpc  | [INFO] [07-04|10:28:49.783] [3/16 Batches] Started downloading L2Blocks routine
erigon-rpc  | [EROR] [07-04|10:28:49.783] [3/16 Batches] Error downloading blocks from datastream error="15.204.143.188:42890 35.189.86.110:6900 Error sending to server: write tcp 15.204.143.188:42890->35.189.86.110:6900: write: broken pipe"
erigon-rpc  | [INFO] [07-04|10:28:49.783] [3/16 Batches] Finished downloading L2Blocks routine
erigon-rpc  | [INFO] [07-04|10:28:49.783] [3/16 Batches] Finished Batches stage
erigon-rpc  | [INFO] [07-04|10:28:49.783] [4/16 BlockHashes] Etl transform started
erigon-rpc  | [INFO] [07-04|10:28:49.783] [4/16 BlockHashes] Nothing new to transform
erigon-rpc  | [INFO] [07-04|10:28:49.783] [4/16 BlockHashes] Etl transform ended
erigon-rpc  | [INFO] [07-04|10:28:49.783] [6/16 Execution] Blocks execution        from=0 to=0
erigon-rpc  | [INFO] [07-04|10:28:49.783] [6/16 Execution] Completed on            block=0
erigon-rpc  | [INFO] [07-04|10:28:49.783] [7/16 CumulativeIndex] Started
erigon-rpc  | [INFO] [07-04|10:28:49.783] [7/16 CumulativeIndex] Nothing new to process
erigon-rpc  | [INFO] [07-04|10:28:49.783] [7/16 CumulativeIndex] Finished
erigon-rpc  | [INFO] [07-04|10:28:49.783] [8/16 HashState] Started
erigon-rpc  | [INFO] [07-04|10:28:49.783] [8/16 HashState] Nothing new to process
erigon-rpc  | [INFO] [07-04|10:28:49.783] [8/16 HashState] Finished
erigon-rpc  | [INFO] [07-04|10:28:49.783] [10/16 CallTraces] Started
erigon-rpc  | [INFO] [07-04|10:28:49.783] [10/16 CallTraces] Nothing new to process
erigon-rpc  | [INFO] [07-04|10:28:49.783] [10/16 CallTraces] Finished
erigon-rpc  | [INFO] [07-04|10:28:49.783] [11/16 AccountHistoryIndex] Started
erigon-rpc  | [INFO] [07-04|10:28:49.783] [11/16 AccountHistoryIndex] Nothing new to process
erigon-rpc  | [INFO] [07-04|10:28:49.783] [11/16 AccountHistoryIndex] Finished
erigon-rpc  | [INFO] [07-04|10:28:49.783] [12/16 StorageHistoryIndex] Started
erigon-rpc  | [INFO] [07-04|10:28:49.783] [12/16 StorageHistoryIndex] Nothing new to process
erigon-rpc  | [INFO] [07-04|10:28:49.783] [12/16 StorageHistoryIndex] Finished
erigon-rpc  | [INFO] [07-04|10:28:49.783] [13/16 LogIndex] Started
erigon-rpc  | [INFO] [07-04|10:28:49.783] [13/16 LogIndex] Nothing new to process
erigon-rpc  | [INFO] [07-04|10:28:49.783] [13/16 LogIndex] Finished
erigon-rpc  | [INFO] [07-04|10:28:49.784] [15/16 DataStream] Starting...
erigon-rpc  | [INFO] [07-04|10:28:49.784] [15/16 DataStream] no streamer provided, skipping stage
erigon-rpc  | [INFO] [07-04|10:28:49.784] [16/16 Finish] Started
erigon-rpc  | [INFO] [07-04|10:28:49.784] [16/16 Finish] Nothing new to process
erigon-rpc  | [INFO] [07-04|10:28:49.784] [16/16 Finish] Finished
Zheaoli commented 5 days ago

The same error

Zheaoli commented 5 days ago

cc @mandrigin

mandrigin commented 5 days ago

polygon is fixing it now, issue is with the sequencer, not erigon

gathecageorge commented 5 days ago

Can you provide reference on where the issue is been fixed so we can follow it?

mandrigin commented 5 days ago

it's a devops task I don't know if it is open to anyone.

but I can write in this issue as soon as i know they fixed it

gathecageorge commented 5 days ago

For me I dont know if its similar but seem like, this

zkevm.l2-datastreamer-url: stream.zkevm-rpc.com:6900

is not reachable and getting error logs like below

[EROR] [07-04|12:47:45.105] Error while executing stage err="[3/16 Batches] l2blocks download routine error: 172.18.0.19:49704 35.189.86.110:6900 Error sending to server: write tcp 172.18.0.19:49704->35.189.86.110:6900: write: broken pipe"

checked and this stream.zkevm-rpc.com is resolving to this ip 35.189.86.110

mandrigin commented 5 days ago

should be fixed in ~30-60 minutes from this message

mandrigin commented 5 days ago

okay, should be fixed now

gathecageorge commented 5 days ago

Getting connnection refused now, not pipe broken

mandrigin commented 5 days ago

@gathecageorge still?

gathecageorge commented 5 days ago

back to broken pipe

zkevm-erigon-1  | [EROR] [07-04|15:12:50.433] Error while executing stage              err="[3/16 Batches] l2blocks download routine error: 172.18.0.19:51994 35.189.86.110:6900 Error sending to server: write tcp 172.18.0.19:51994->35.189.86.110:6900: write: broken pipe"
zkevm-erigon-1  | [EROR] [07-04|15:12:50.433] Staged Sync                              err="[3/16 Batches] l2blocks download routine error: 172.18.0.19:51994 35.189.86.110:6900 Error sending to server: write tcp 172.18.0.19:51994->35.189.86.110:6900: write: broken pipe"
zkevm-erigon-1  | [INFO] [07-04|15:12:51.198] [3/16 Batches] Starting batches stage 
zkevm-erigon-1  | [INFO] [07-04|15:12:51.198] [3/16 Batches] Starting stream           startBlock=13908563
zkevm-erigon-1  | [INFO] [07-04|15:12:51.198] [3/16 Batches] Reading blocks from the datastream. 
zkevm-erigon-1  | [INFO] [07-04|15:12:51.198] [3/16 Batches] Started downloading L2Blocks routine 
zkevm-erigon-1  | [INFO] [07-04|15:12:51.198] [3/16 Batches] Finished Batches stage 
zkevm-erigon-1  | [EROR] [07-04|15:12:51.198] Error while executing stage              err="[3/16 Batches] l2blocks download routine error: 172.18.0.19:51994 35.189.86.110:6900 Error sending to server: write tcp 172.18.0.19:51994->35.189.86.110:6900: write: broken pipe"
zkevm-erigon-1  | [INFO] [07-04|15:12:51.198] [3/16 Batches] Finished downloading L2Blocks routine 
zkevm-erigon-1  | [EROR] [07-04|15:12:51.198] Staged Sync                              err="[3/16 Batches] l2blocks download routine error: 172.18.0.19:51994 35.189.86.110:6900 Error sending to server: write tcp 172.18.0.19:51994->35.189.86.110:6900: write: broken pipe"
mandrigin commented 5 days ago

very strange, I can sync my node just fine right now :-/

gathecageorge commented 5 days ago

which version are you using? and your configuration

Borman322 commented 5 days ago

Can u provide about your version and config file? Bacause i got broken pipe too

gathecageorge commented 5 days ago

Version v1.0.10, providing the --zkevm.l1-rpc-url ${L1_RPC} via arguments

datadir: /home/erigon/.local/share/erigon
chain: hermez-mainnet
zkevm.l2-chain-id: 1101
zkevm.l2-sequencer-rpc-url: https://zkevm-rpc.com
zkevm.l2-datastreamer-url: stream.zkevm-rpc.com:6900
zkevm.l1-chain-id: 1

zkevm.address-sequencer: "0x148Ee7dAF16574cD020aFa34CC658f8F3fbd2800"
zkevm.address-zkevm: "0x519E42c24163192Dca44CD3fBDCEBF6be9130987"
zkevm.address-admin: "0x242daE44F5d8fb54B198D03a94dA45B5a4413e21"
zkevm.address-rollup: "0x5132A183E9F3CB7C848b0AAC5Ae0c4f0491B7aB2"
zkevm.address-ger-manager: "0x580bda1e7A0CFAe92Fa7F6c20A3794F169CE3CFb"

zkevm.default-gas-price: 1
zkevm.max-gas-price: 0
zkevm.gas-price-factor: 0.0375

zkevm.l1-rollup-id: 1
zkevm.l1-block-range: 20000
zkevm.l1-query-delay: 6000
zkevm.l1-first-block: 16896700
zkevm.datastream-version: 2

externalcl: true
mandrigin commented 5 days ago

ah, okay, please use https://github.com/0xPolygonHermez/cdk-erigon/releases/tag/v1.1.5.5 for mainnet now