search

erigontech / erigon

Ethereum implementation on the efficiency frontier https://erigon.gitbook.io
GNU Lesser General Public License v3.0
3.13k stars 1.11k forks source link

Polygon Amoy snapshot download standstill #11436

Open MrFreezeDZ opened 2 months ago

MrFreezeDZ commented 2 months ago

System information

Erigon version: https://hub.docker.com/layers/thorax/erigon/2.60.5-amd64/images/sha256-199f46cb03573a6d1d0b296a73c252d2950a6d8eba8cf0526f0fc283d23be1a3?context=explore

OS & Version: Windows/Linux/OSX Google Kubernetes Engine https://hub.docker.com/layers/thorax/erigon/2.60.5-amd64/images/sha256-199f46cb03573a6d1d0b296a73c252d2950a6d8eba8cf0526f0fc283d23be1a3?context=explore

Erigon Command (with flags/config): erigon --chain=amoy --datadir=/data/ --log.json=true --http.addr=0.0.0.0 --http.vhosts= --http --http.api=eth,debug,trace --ws --bor.heimdall=http://heimdallrest --authrpc.vhosts= --authrpc.jwtsecret=/secret/jwt.hex --authrpc.addr=0.0.0.0 --db.size.limit=2TB --db.pagesize=16k --metrics --metrics.addr=0.0.0.0 --maxpeers=500 --torrent.download.rate=300mb

Consensus Layer: heimdall 1.0.7

Chain/Network: amoy

Expected behaviour

Snapshot download will finish.

Actual behaviour

Snapshot download keeps logging 99.90% progress for hours and no restart helps anything.

Steps to reproduce the behaviour

I am not sure how to reproduce the behaviour on purpose. We did just start a new Erigon instance for the polygon amoy testnet.

Backtrace

These are the lines that get logged again and again.

{"Anonymous":"161.2MB", "PrivateClean":"63.8MB", "PrivateDirty":"161.3MB", "Pss":"225.4MB", "Referenced":"225.6MB", "Rss":"225.6MB", "SharedClean":"504.0KB", "SharedDirty":"0B", "Size":"0B", "Swap":"0B", "alloc":"86.6MB", "lvl":"info", "msg":"[mem] memory stats", "sys":"168.3MB", "t":"2024-08-01T08:57:38.822929588Z"}
{"alloc":"86.6MB", "baseFee":0, "lvl":"info", "msg":"[txpool] stat", "pending":0, "queued":0, "sys":"168.3MB", "t":"2024-08-01T08:57:38.852932237Z"}
{"alloc":"87.1MB", "connections":0, "download":"0B/s", "files":405, "lvl":"info", "metadata":"401/405", "msg":"[1/15 Snapshots] download", "peers":0, "progress":"99.90% 6.7GB/6.7GB", "sys":"168.3MB", "t":"2024-08-01T08:57:38.983239651Z", "time-left":"999hrs:99m", "total-time":"9m0s", "upload":"0B/s"}
{"files":4, "list":"v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006300-006400-borevents.seg,v1-006400-006500-borevents.seg", "lvl":"info", "msg":"[snapshots] no metadata yet", "t":"2024-08-01T08:57:58.820797Z"}
{"alloc":"86.4MB", "connections":0, "download":"0B/s", "files":405, "lvl":"info", "metadata":"401/405", "msg":"[1/15 Snapshots] download", "peers":0, "progress":"99.90% 6.7GB/6.7GB", "sys":"168.3MB", "t":"2024-08-01T08:57:58.983374313Z", "time-left":"999hrs:99m", "total-time":"9m20s", "upload":"0B/s"}
MrFreezeDZ commented 2 months ago

As in Discord suggested we stopped Erigon and deleted the downloader folder inside of Erigon's datadir with the following command:

rm -rf downloader

These are the logs we then saw, after we restarted Erigon again:

{"file prefix":"erigon","json":false,"log dir":"/data/logs","log level":"info","lvl":"info","msg":"logging to file system","t":"2024-08-02T12:37:48.023562884Z"}
{"lvl":"info","msg":"Enabling metrics export to prometheus","path":"http://0.0.0.0:6060/debug/metrics/prometheus","t":"2024-08-02T12:37:48.029207523Z"}
{"git_branch":"","git_commit":"","git_tag":"","lvl":"info","msg":"Build info","t":"2024-08-02T12:37:48.029262934Z"}
{"lvl":"info","msg":"Starting Erigon on Amoy testnet...","t":"2024-08-02T12:37:48.029305834Z"}
{"ETH":500,"lvl":"info","msg":"Maximum peer count","t":"2024-08-02T12:37:48.035552643Z","total":500}
{"APIs":"eth,debug,trace","lvl":"info","msg":"starting HTTP APIs","port":8545,"t":"2024-08-02T12:37:48.035744023Z"}
{"level":"WRN","lvl":"info","msg":"torrent verbosity","t":"2024-08-02T12:37:48.036033373Z"}
{"cap":50000000,"lvl":"info","msg":"Set global gas cap","t":"2024-08-02T12:37:50.140481571Z"}
{"download.rate":"300mb","ipv4-enabled":true,"ipv6-enabled":true,"lvl":"info","msg":"[Downloader] Running with","t":"2024-08-02T12:37:50.142293371Z","upload.rate":"4mb"}
{"label":"chaindata","lvl":"info","msg":"Opening Database","path":"/data/chaindata","t":"2024-08-02T12:37:50.142455271Z"}
{"label":"chaindata","lvl":"info","msg":"[db] open","pageSize":16384,"sizeLimit":"2TB","t":"2024-08-02T12:37:50.147183211Z"}
{"lvl":"warn","msg":"Sanitizing invalid bor miner gas price","provided":"1000000000","t":"2024-08-02T12:37:50.15376389Z","updated":"25000000000"}
{"lvl":"warn","msg":"Sanitizing invalid bor min fee cap","provided":1,"t":"2024-08-02T12:37:50.153817101Z","updated":25000000000}
{"config":"{ChainID: 80002, Homestead: 0, DAO: \u003cnil\u003e, Tangerine Whistle: 0, Spurious Dragon: 0, Byzantium: 0, Constantinople: 0, Petersburg: 0, Istanbul: 0, Muir Glacier: 0, Berlin: 0, London: 73100, Arrow Glacier: \u003cnil\u003e, Gray Glacier: \u003cnil\u003e, Terminal Total Difficulty: \u003cnil\u003e, Merge Netsplit: \u003cnil\u003e, Shanghai: \u003cnil\u003e, Cancun: \u003cnil\u003e, Prague: \u003cnil\u003e, Osaka: \u003cnil\u003e, Engine: bor, NoPruneContracts: map[]}","genesis":"0x7202b2b53c5a0836e773e319d18922cc756dd67432f9a1f65352b61f4406c697","lvl":"info","msg":"Initialised chain configuration","t":"2024-08-02T12:37:50.15386839Z"}
{"lvl":"info","msg":"Initialising Ethereum protocol","network":80002,"t":"2024-08-02T12:38:25.881509529Z"}
{"label":"bor","lvl":"info","msg":"Opening Database","path":"/data/bor","t":"2024-08-02T12:38:25.881637039Z"}
{"lvl":"info","msg":"Starting private RPC server","on":"127.0.0.1:9090","t":"2024-08-02T12:38:26.1172805Z"}
{"lvl":"info","msg":"rpc filters: subscribing to Erigon events","t":"2024-08-02T12:38:26.11761547Z"}
{"lvl":"info","msg":"new subscription to logs established","t":"2024-08-02T12:38:26.143098378Z"}
{"lvl":"info","msg":"[txpool] Started","t":"2024-08-02T12:38:26.143662048Z"}
{"lvl":"info","msg":"new subscription to newHeaders established","t":"2024-08-02T12:38:26.143758938Z"}
{"lvl":"info","msg":"New txs subscriber joined","t":"2024-08-02T12:38:26.143892858Z"}
{"grpc":false,"http.url":"[::]:8545","lvl":"info","msg":"JsonRpc endpoint opened","t":"2024-08-02T12:38:26.144893297Z","websocket.url":"[::]:8546","ws":true,"ws.compression":true}
{"lvl":"info","msg":"[1/15 Snapshots] Requesting downloads","t":"2024-08-02T12:38:26.575334513Z"}
{"err":"no UPnP or NAT-PMP router discovered","lvl":"warn","msg":"NAT ExternalIP resolution has failed, try to pass a different --nat option","t":"2024-08-02T12:38:26.661465906Z"}
{"lvl":"info","msg":"Started P2P networking","name":"erigon/v2.60.5/linux-amd64/go1.21.6","self":"enode://efc37240445a474f00f42133d163b37268e65c7271922813dec962e2dd4d21b8016c80b1bd320c1b1427defe4076c2f606bb0c2879e45195e5f23a5799eef198@127.0.0.1:30304","t":"2024-08-02T12:38:26.669585025Z","version":67}
{"err":"no UPnP or NAT-PMP router discovered","lvl":"warn","msg":"NAT ExternalIP resolution has failed, try to pass a different --nat option","t":"2024-08-02T12:38:26.693249093Z"}
{"lvl":"info","msg":"Started P2P networking","name":"erigon/v2.60.5/linux-amd64/go1.21.6","self":"enode://efc37240445a474f00f42133d163b37268e65c7271922813dec962e2dd4d21b8016c80b1bd320c1b1427defe4076c2f606bb0c2879e45195e5f23a5799eef198@127.0.0.1:30303","t":"2024-08-02T12:38:26.737765279Z","version":68}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:38:27.519528436Z"}
{"files":16,"list":"v1-007900-008000-bodies.seg,v1-007900-008000-transactions.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:38:27.519981105Z"}
{"files":4,"list":"v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:38:45.61982261Z"}
{"files":13,"list":"v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:38:45.61989516Z"}
{"alloc":"56.6MB","connections":0,"download":"1.4MB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"97.25% 7.2GB/7.4GB","sys":"183.3MB","t":"2024-08-02T12:38:47.520494005Z","time-left":"0hrs:2m","total-time":"20s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:39:05.61971668Z"}
{"files":13,"list":"v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:39:05.61982802Z"}
{"alloc":"54.6MB","connections":0,"download":"3.1MB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"183.7MB","t":"2024-08-02T12:39:07.520931125Z","time-left":"0hrs:0m","total-time":"40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:39:25.620766399Z"}
{"files":13,"list":"v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,v1-008100-008200-borspans.seg,v1-008500-008600-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:39:25.620845579Z"}
{"alloc":"54.2MB","connections":0,"download":"1.5MB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"183.8MB","t":"2024-08-02T12:39:27.521299784Z","time-left":"0hrs:1m","total-time":"1m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:39:45.619727699Z"}
{"files":13,"list":"v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:39:45.619802199Z"}
{"alloc":"66.2MB","connections":0,"download":"789.0KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.0MB","t":"2024-08-02T12:39:47.520984834Z","time-left":"0hrs:3m","total-time":"1m20s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:40:05.618925939Z"}
{"files":13,"list":"v1-008100-008200-borspans.seg,v1-008500-008600-borspans.seg,v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:40:05.619006249Z"}
{"alloc":"44.3MB","connections":0,"download":"394.5KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.0MB","t":"2024-08-02T12:40:07.520841294Z","time-left":"0hrs:6m","total-time":"1m40s","upload":"0B/s"}
{"files":4,"list":"v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:40:25.619664325Z"}
{"files":13,"list":"v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:40:25.619741755Z"}
{"alloc":"57.7MB","connections":0,"download":"197.2KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.0MB","t":"2024-08-02T12:40:27.520669526Z","time-left":"0hrs:12m","total-time":"2m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:40:45.620221594Z"}
{"files":13,"list":"v1-008500-008600-borspans.seg,v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:40:45.620296204Z"}
{"alloc":"57.1MB","connections":0,"download":"98.6KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.3MB","t":"2024-08-02T12:40:47.520706685Z","time-left":"0hrs:25m","total-time":"2m20s","upload":"0B/s"}
{"files":4,"list":"v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:41:05.619330883Z"}
{"files":13,"list":"v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:41:05.619428723Z"}
{"alloc":"97.8MB","connections":0,"download":"49.3KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.3MB","t":"2024-08-02T12:41:07.520363184Z","time-left":"0hrs:50m","total-time":"2m40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:41:25.61989552Z"}
{"files":13,"list":"v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:41:25.61997406Z"}
{"lvl":"info","msg":"[p2p] GoodPeers","t":"2024-08-02T12:41:25.907772797Z"}
{"Anonymous":"172.6MB","PrivateClean":"7.5GB","PrivateDirty":"172.7MB","Pss":"7.7GB","Referenced":"7.7GB","Rss":"7.7GB","SharedClean":"4.0KB","SharedDirty":"0B","Size":"0B","Swap":"0B","alloc":"53.2MB","lvl":"info","msg":"[mem] memory stats","sys":"184.3MB","t":"2024-08-02T12:41:25.995285741Z"}
{"alloc":"53.6MB","baseFee":0,"lvl":"info","msg":"[txpool] stat","pending":0,"queued":0,"sys":"184.3MB","t":"2024-08-02T12:41:26.143682979Z"}
{"alloc":"57.1MB","connections":0,"download":"24.7KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.3MB","t":"2024-08-02T12:41:27.52146362Z","time-left":"1hrs:41m","total-time":"3m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:41:45.619266647Z"}
{"files":13,"list":"v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:41:45.619342337Z"}
{"alloc":"91.1MB","connections":0,"download":"12.3KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.5MB","t":"2024-08-02T12:41:47.521113768Z","time-left":"3hrs:23m","total-time":"3m20s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:42:05.620121244Z"}
{"files":13,"list":"v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:42:05.620209493Z"}
{"alloc":"86.6MB","connections":0,"download":"6.2KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:42:07.520705864Z","time-left":"6hrs:47m","total-time":"3m40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:42:25.620005621Z"}
{"files":13,"list":"v1-008100-008200-borspans.seg,v1-008500-008600-borspans.seg,v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:42:25.620085721Z"}
{"alloc":"89.8MB","connections":0,"download":"3.1KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:42:27.520635531Z","time-left":"13hrs:35m","total-time":"4m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:42:45.62030553Z"}
{"files":13,"list":"v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:42:45.62038504Z"}
{"alloc":"76.2MB","connections":0,"download":"1.5KB/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:42:47.521211621Z","time-left":"27hrs:12m","total-time":"4m20s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:43:05.619599597Z"}
{"files":13,"list":"v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:43:05.619823987Z"}
{"alloc":"69.0MB","connections":0,"download":"788B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:43:07.520720247Z","time-left":"54hrs:26m","total-time":"4m40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:43:25.619142151Z"}
{"files":13,"list":"v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:43:25.619231011Z"}
{"alloc":"79.5MB","connections":0,"download":"394B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:43:27.520536522Z","time-left":"108hrs:52m","total-time":"5m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:43:45.620449527Z"}
{"files":13,"list":"v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:43:45.620513967Z"}
{"alloc":"64.1MB","connections":0,"download":"197B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:43:47.521288488Z","time-left":"217hrs:45m","total-time":"5m20s","upload":"0B/s"}
{"files":4,"list":"v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:44:05.619381696Z"}
{"files":13,"list":"v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:44:05.619473516Z"}
{"alloc":"64.5MB","connections":0,"download":"98B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:44:07.520560607Z","time-left":"437hrs:44m","total-time":"5m40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:44:25.619703366Z"}
{"files":13,"list":"v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:44:25.619786996Z"}
{"lvl":"info","msg":"[p2p] GoodPeers","t":"2024-08-02T12:44:25.907499723Z"}
{"Anonymous":"175.1MB","PrivateClean":"7.5GB","PrivateDirty":"175.1MB","Pss":"7.7GB","Referenced":"7.7GB","Rss":"7.7GB","SharedClean":"4.0KB","SharedDirty":"0B","Size":"0B","Swap":"0B","alloc":"84.1MB","lvl":"info","msg":"[mem] memory stats","sys":"184.8MB","t":"2024-08-02T12:44:26.019082834Z"}
{"alloc":"84.5MB","baseFee":0,"lvl":"info","msg":"[txpool] stat","pending":0,"queued":0,"sys":"184.8MB","t":"2024-08-02T12:44:26.143916835Z"}
{"alloc":"87.9MB","connections":0,"download":"49B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:44:27.520906096Z","time-left":"875hrs:29m","total-time":"6m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:44:45.620228514Z"}
{"files":13,"list":"v1-008500-008600-borspans.seg,v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:44:45.620315754Z"}
{"alloc":"68.0MB","connections":0,"download":"24B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:44:47.521280134Z","time-left":"1787hrs:28m","total-time":"6m20s","upload":"0B/s"}
{"files":4,"list":"v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:45:05.619472177Z"}
{"files":13,"list":"v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:45:05.619539937Z"}
{"alloc":"93.6MB","connections":0,"download":"12B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:45:07.520746767Z","time-left":"3574hrs:56m","total-time":"6m40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:45:25.619273893Z"}
{"files":13,"list":"v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:45:25.619353223Z"}
{"alloc":"73.7MB","connections":0,"download":"6B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:45:27.521006003Z","time-left":"7149hrs:53m","total-time":"7m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:45:45.620262959Z"}
{"files":13,"list":"v1-008100-008200-borspans.seg,v1-008500-008600-borspans.seg,v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:45:45.620344809Z"}
{"alloc":"53.1MB","connections":0,"download":"3B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:45:47.521379019Z","time-left":"14299hrs:47m","total-time":"7m20s","upload":"0B/s"}
{"files":4,"list":"v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:46:05.618939803Z"}
{"files":13,"list":"v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:46:05.619011503Z"}
{"alloc":"81.0MB","connections":0,"download":"1B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:46:07.520474503Z","time-left":"42899hrs:23m","total-time":"7m40s","upload":"0B/s"}
{"files":4,"list":"v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:46:25.619448286Z"}
{"files":13,"list":"v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:46:25.619533526Z"}
{"alloc":"52.2MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:46:27.520715766Z","time-left":"999hrs:99m","total-time":"8m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:46:45.61972936Z"}
{"files":13,"list":"v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,v1-008100-008200-borspans.seg,v1-008500-008600-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:46:45.61980233Z"}
{"alloc":"89.5MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:46:47.521328219Z","time-left":"999hrs:99m","total-time":"8m20s","upload":"0B/s"}
{"files":4,"list":"v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:47:05.620773261Z"}
{"files":13,"list":"v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:47:05.620854001Z"}
{"alloc":"65.3MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:47:07.520683901Z","time-left":"999hrs:99m","total-time":"8m40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:47:25.619955183Z"}
{"files":13,"list":"v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:47:25.620038753Z"}
{"lvl":"info","msg":"[p2p] GoodPeers","t":"2024-08-02T12:47:25.908103831Z"}
{"Anonymous":"174.7MB","PrivateClean":"7.5GB","PrivateDirty":"174.9MB","Pss":"7.7GB","Referenced":"7.7GB","Rss":"7.7GB","SharedClean":"4.0KB","SharedDirty":"0B","Size":"0B","Swap":"0B","alloc":"75.3MB","lvl":"info","msg":"[mem] memory stats","sys":"184.8MB","t":"2024-08-02T12:47:25.994976324Z"}
{"alloc":"75.9MB","baseFee":0,"lvl":"info","msg":"[txpool] stat","pending":0,"queued":0,"sys":"184.8MB","t":"2024-08-02T12:47:26.143663092Z"}
{"alloc":"85.2MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:47:27.520739963Z","time-left":"999hrs:99m","total-time":"9m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:47:45.620429707Z"}
{"files":13,"list":"v1-008500-008600-borspans.seg,v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:47:45.620507918Z"}
{"alloc":"75.2MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:47:47.520485507Z","time-left":"999hrs:99m","total-time":"9m20s","upload":"0B/s"}
{"files":4,"list":"v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg,v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:48:05.619646208Z"}
{"files":13,"list":"v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:48:05.619719368Z"}
{"alloc":"52.7MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:48:07.520318648Z","time-left":"999hrs:99m","total-time":"9m40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:48:25.619570916Z"}
{"files":13,"list":"v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:48:25.619665646Z"}
{"alloc":"70.1MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:48:27.520708576Z","time-left":"999hrs:99m","total-time":"10m0s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:48:45.621941408Z"}
{"files":13,"list":"v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,v1-008300-008400-borevents.seg,v1-008100-008200-borspans.seg,v1-008500-008600-borspans.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:48:45.622110228Z"}
{"alloc":"76.2MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:48:47.521005709Z","time-left":"999hrs:99m","total-time":"10m20s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:49:05.620620707Z"}
{"files":13,"list":"v1-008400-008500-borevents.seg,v1-008700-008800-transactions.seg,v1-008800-008900-transactions.seg,v1-006400-006500-borspans.seg,v1-008600-008700-transactions.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:49:05.620705497Z"}
{"alloc":"49.7MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:49:07.520350418Z","time-left":"999hrs:99m","total-time":"10m40s","upload":"0B/s"}
{"files":4,"list":"v1-008000-008100-borevents.seg,v1-006400-006500-borevents.seg,v1-006300-006400-borevents.seg,v1-007900-008000-borevents.seg","lvl":"info","msg":"[snapshots] no metadata yet","t":"2024-08-02T12:49:25.619707155Z"}
{"files":13,"list":"v1-008300-008400-borspans.seg,v1-008000-008100-borspans.seg,v1-006300-006400-borspans.seg,v1-007900-008000-borspans.seg,v1-008500-008600-borevents.seg,...","lvl":"info","msg":"[snapshots] no progress yet","t":"2024-08-02T12:49:25.619794564Z"}
{"alloc":"66.0MB","connections":0,"download":"0B/s","files":445,"lvl":"info","metadata":"441/445","msg":"[1/15 Snapshots] download","peers":0,"progress":"98.06% 7.3GB/7.4GB","sys":"184.8MB","t":"2024-08-02T12:49:27.521377705Z","time-left":"999hrs:99m","total-time":"11m0s","upload":"0B/s"}