search

celestiaorg / celestia-node

Celestia Data Availability Nodes
Apache License 2.0
923 stars 919 forks source link

bug: node starts on mainnet with mocha core.ip, with seemingly no errors #3597

Open jcstein opened 1 month ago

jcstein commented 1 month ago

Celestia Node version

v0.14.0

OS

macos

Install tools

docs for v0.14.0

Others

No response

Steps to reproduce it

run light node on mainnet with mocha core.ip endpoint

Expected result

some kind of failure, e.g. WRONG NETWORK PROVIDED IN CORE.IP URL

Actual result

see logs below or in warp https://app.warp.dev/block/PMZu7uBSX6vMLrs8xzQveH

Relevant log output

$ celestia light start --core.ip rpc-mocha.pops.one
context menu

2024-07-25T14:34:59.390-0400    INFO    node    nodebuilder/module.go:26    Accessing keyring...
2024-07-25T14:34:59.425-0400    INFO    badger4 v4@v4.2.1-0.20240106094458-1c417aa3799c/levels.go:171   All 0 tables opened in 0s

2024-07-25T14:34:59.431-0400    INFO    badger4 v4@v4.2.1-0.20240106094458-1c417aa3799c/discard.go:66   Discard stats nextEmptySlot: 0

2024-07-25T14:34:59.431-0400    INFO    badger4 v4@v4.2.1-0.20240106094458-1c417aa3799c/db.go:368   Set nextTxnTs to 0
2024-07-25T14:35:00.144-0400    INFO    pidstore    pidstore/pidstore.go:84 Persisted peers successfully    {"amount": 0}
2024-07-25T14:35:00.144-0400    INFO    module/header   header/config.go:66 No trusted peers in config, initializing with default bootstrappers as trusted peers
2024-07-25T14:35:00.150-0400    INFO    header/p2p  p2p/exchange.go:99  client: starting client {"protocol ID": "/celestia/header-ex/v0.0.3"}
2024-07-25T14:35:00.150-0400    INFO    pidstore    pidstore/pidstore.go:67 Loaded peers from disk  {"amount": 0}
2024-07-25T14:35:01.122-0400    INFO    header/store    store/store.go:139  initialized head    {"height": 1, "hash": "6BE39EFD10BA412A9DB5288488303F5DD32CF386707A5BEF33617F4C43301872"}
2024-07-25T14:35:01.122-0400    INFO    header/p2p  p2p/subscriber.go:80    joining topic   {"topic ID": "/celestia/header-sub/v0.0.1"}
2024-07-25T14:35:01.122-0400    INFO    fraudserv   fraudserv/helpers.go:29 joining topic   {"id": "/celestia/fraud-sub/badencodingv0.1/v0.0.1"}
2024-07-25T14:35:01.122-0400    INFO    fraudserv   fraudserv/service.go:114    starting fraud proof service    {"protocol ID": "/celestia/fraud/v0.0.1"}
2024-07-25T14:35:02.313-0400    WARN    state   state/core_access.go:181    failed to set up signer, check if node's account is funded  {"err": "rpc error: code = NotFound desc = account celestia1j6ppzj5e3xkz8d22ys4qyz55ddwtxttu6np72y not found"}
2024-07-25T14:35:02.444-0400    INFO    share/discovery discovery/discovery.go:123  started discovery   {"topic": "full"}
2024-07-25T14:35:02.444-0400    INFO    share/discovery discovery/discovery.go:123  started discovery   {"topic": "archival"}
2024-07-25T14:35:02.445-0400    INFO    header/sync sync/sync_head.go:82    stored head header expired  {"height": 1}
2024-07-25T14:35:02.702-0400    INFO    header/sync sync/sync_head.go:140   new network head    {"height": 1970736, "hash": "BFA589E8B345E3C294FF188FB29711585EE70071DD7E36549F9C38A98BC7449D"}
2024-07-25T14:35:02.702-0400    WARN    header/sync sync/sync_head.go:108   subjective initialization with an old header    {"height": 1970736}
2024-07-25T14:35:02.702-0400    WARN    header/sync sync/sync_head.go:110   trusted peer is out of sync
2024-07-25T14:35:02.867-0400    WARN    header/sync sync/sync_head.go:46    failed to get recent head, returning current subjective {"sbjHead": 1970736, "err": "header: not found"}
2024-07-25T14:35:02.868-0400    INFO    header/sync sync/sync.go:216    syncing headers {"from": 2, "to": 1970736}
2024-07-25T14:35:02.868-0400    INFO    header/p2p  p2p/server.go:77    server: listening for inbound header requests   {"protocol ID": "/celestia/header-ex/v0.0.3"}
2024-07-25T14:35:02.868-0400    WARN    das das/daser.go:100    checkpoint not found, initializing with height 1
2024-07-25T14:35:02.868-0400    INFO    das das/daser.go:113    starting DASer from checkpoint: SampleFrom: 1, NetworkHead: 1
2024-07-25T14:35:02.868-0400    INFO    rpc rpc/server.go:90    server started  {"listening on": "127.0.0.1:26658"}
2024-07-25T14:35:02.868-0400    INFO    node    nodebuilder/node.go:117 

/_____/  /_____/  /_____/  /_____/  /_____/

Started celestia DA node
node version:   v0.14.0
node type:  light
network:    celestia

/_____/  /_____/  /_____/  /_____/  /_____/

The p2p host is listening on:
*  /ip4/10.0.0.155/tcp/2121/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip4/10.0.0.155/udp/2121/quic-v1/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip4/10.0.0.155/udp/2121/quic-v1/webtransport/certhash/uEiAoy0MVkLBfAg-ImgYhORzmLntmIDDUFPxRVyq0Q3YdlA/certhash/uEiBaJBBcSJj7HXEMK5Bivnya53GRqGRPIWp8dPGkCWCF3w/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip4/71.200.65.106/tcp/2121/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip4/71.200.65.106/udp/2121/quic-v1/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip4/71.200.65.106/udp/2121/quic-v1/webtransport/certhash/uEiAoy0MVkLBfAg-ImgYhORzmLntmIDDUFPxRVyq0Q3YdlA/certhash/uEiBaJBBcSJj7HXEMK5Bivnya53GRqGRPIWp8dPGkCWCF3w/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip4/127.0.0.1/udp/2121/quic-v1/webtransport/certhash/uEiAoy0MVkLBfAg-ImgYhORzmLntmIDDUFPxRVyq0Q3YdlA/certhash/uEiBaJBBcSJj7HXEMK5Bivnya53GRqGRPIWp8dPGkCWCF3w/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip6/::1/tcp/2121/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip6/::1/udp/2121/quic-v1/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ
*  /ip6/::1/udp/2121/quic-v1/webtransport/certhash/uEiAoy0MVkLBfAg-ImgYhORzmLntmIDDUFPxRVyq0Q3YdlA/certhash/uEiBaJBBcSJj7HXEMK5Bivnya53GRqGRPIWp8dPGkCWCF3w/p2p/12D3KooWMzzxGKCqgsRjjJcKd9at2zSwUCd7s59We69vKzJD2tJQ

2024-07-25T14:35:02.868-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1, "to": 1, "errors": 0, "# of headers skipped as outside of sampling window": 1, "finished (s)": 0.000008709}
2024-07-25T14:35:03.594-0400    INFO    share/discovery discovery/discovery.go:325  discovered wanted peers {"topic": "full", "amount": 5}
2024-07-25T14:35:03.662-0400    INFO    share/discovery discovery/discovery.go:325  discovered wanted peers {"topic": "archival", "amount": 5}
2024-07-25T14:35:04.361-0400    INFO    header/sync sync/sync_head.go:140   new network head    {"height": 1970737, "hash": "9CE1E6A91A314560CB530F7EE9A843B4418EA4F6E00355421FECE67E7A3D9AE4"}
2024-07-25T14:35:04.361-0400    INFO    das das/state.go:146    found first header, starting sampling
2024-07-25T14:35:09.602-0400    INFO    das das/worker.go:161   sampled header  {"type": "recent", "height": 1970737, "hash": "9CE1E6A91A314560CB530F7EE9A843B4418EA4F6E00355421FECE67E7A3D9AE4", "square width": 32, "data root": "265431C5EE4F9A196AD114F669A4BD8A6CF19090968F5E1193469921C56B044A", "finished (s)": 5.241181875}
2024-07-25T14:35:10.314-0400    INFO    header/sync sync/sync_head.go:140   new network head    {"height": 1970738, "hash": "18328744DE754BC7B13C9958B0407E3EC9ACCB1AE49BFA9084859700B0F2B559"}
2024-07-25T14:35:10.655-0400    INFO    das das/worker.go:161   sampled header  {"type": "recent", "height": 1970738, "hash": "18328744DE754BC7B13C9958B0407E3EC9ACCB1AE49BFA9084859700B0F2B559", "square width": 4, "data root": "FA7240C4880621260D19B62681EB1B1C3CBD592C99B193BD8D900B857CDAC0BB", "finished (s)": 0.340999708}
2024-07-25T14:35:11.424-0400    ERROR   state   state/core_access.go:355    balance for account celestia1j6ppzj5e3xkz8d22ys4qyz55ddwtxttu6np72y does not exist at block height 1970737
2024-07-25T14:35:11.639-0400    INFO    header/store    store/store.go:367  new head    "height": 513, "hash": "D34533754C7407A3FF14DD3D4044FC08ED9F39884545922DC19D69DCEE498B84"}
2024-07-25T14:35:11.640-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 202, "to": 301, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 7.27847675}
2024-07-25T14:35:11.640-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 402, "to": 501, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 7.278487083}
2024-07-25T14:35:11.640-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 302, "to": 401, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 7.278585125}
2024-07-25T14:35:11.640-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 102, "to": 201, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 7.278663}
2024-07-25T14:35:11.640-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2, "to": 101, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 7.278658542}
2024-07-25T14:35:14.365-0400    WARN    peerstore/ds    pstoreds/addr_book.go:611   Was passed p2p address with a different peerId. found: 12D3KooWBkY9UHLUU4kMkijYi3FoavJTEBLcG2XXXwWucR7b1xuE, expected: 12D3KooWSkkB4XWfWwAFpva2g7uSdoHqWtjVT7QyF72Qs6E5RHdm
2024-07-25T14:35:14.366-0400    WARN    peerstore   pstoremem/addr_book.go:248  Was passed p2p address with a different peerId. found: 12D3KooWBkY9UHLUU4kMkijYi3FoavJTEBLcG2XXXwWucR7b1xuE, expected: 12D3KooWSkkB4XWfWwAFpva2g7uSdoHqWtjVT7QyF72Qs6E5RHdm
2024-07-25T14:35:14.481-0400    INFO    canonical-log   swarm/swarm_listen.go:136   CANONICAL_PEER_STATUS: peer=12D3KooWDprmzQ8uyCGkSykmUzP1RC8VCPvYCYhgSmZtZGtK9WLX addr=/ip4/52.215.16.44/udp/55580/quic-v1 sample_rate=100 connection_status="established" dir="inbound"
2024-07-25T14:35:18.297-0400    INFO    header/store    store/store.go:367  new head    "height": 1025, "hash": "ABE93F125A6D6BF3D8B37C2306B6DB4E14D3B5720E0CCF7C060511F141EE583E"}
2024-07-25T14:35:18.297-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 602, "to": 701, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 13.936383459}
2024-07-25T14:35:18.297-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 802, "to": 901, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 13.936378}
2024-07-25T14:35:18.297-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 502, "to": 601, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 13.936336583}
2024-07-25T14:35:18.297-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 702, "to": 801, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 13.936390709}
2024-07-25T14:35:18.298-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 902, "to": 1001, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 13.936429917}
2024-07-25T14:35:22.635-0400    INFO    header/sync sync/sync_head.go:140   new network head    {"height": 1970739, "hash": "3B85B635686D9A755097840F83FB590A06BC47244ED3D802A42CDAB2FC8CA3FD"}
2024-07-25T14:35:23.078-0400    INFO    das das/worker.go:161   sampled header  {"type": "recent", "height": 1970739, "hash": "3B85B635686D9A755097840F83FB590A06BC47244ED3D802A42CDAB2FC8CA3FD", "square width": 64, "data root": "3198ED241C5921FDBED2CFBD3A21D419BFB528AC87210A011EBE00063733BF0C", "finished (s)": 0.442757584}
2024-07-25T14:35:24.367-0400    INFO    header/store    store/store.go:367  new head    "height": 1537, "hash": "1AEE0A6BB942D9F5DFC23AB1D478AEC4D72AC2DFF5686AE1BCAA9B6DE67760EB"}
2024-07-25T14:35:24.367-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1302, "to": 1401, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 20.006204042}
2024-07-25T14:35:24.368-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1202, "to": 1301, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 20.00656625}
2024-07-25T14:35:24.368-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1002, "to": 1101, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 20.00661875}
2024-07-25T14:35:24.368-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1402, "to": 1501, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 20.006689166}
2024-07-25T14:35:24.368-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1102, "to": 1201, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 20.00667825}
2024-07-25T14:35:30.082-0400    INFO    header/store    store/store.go:367  new head    "height": 2049, "hash": "E4536116A157B22EFC8EFA2CDC4FA0BDF45BFFC3E57E811CE97349350B431A0E"}
2024-07-25T14:35:30.082-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1502, "to": 1601, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 20.479896458}
2024-07-25T14:35:30.082-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1602, "to": 1701, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 18.442743084}
2024-07-25T14:35:30.082-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1902, "to": 2001, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 18.44257025}
2024-07-25T14:35:30.082-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1802, "to": 1901, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 18.442671375}
2024-07-25T14:35:30.083-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 1702, "to": 1801, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 18.442824708}
2024-07-25T14:35:34.654-0400    INFO    header/sync sync/sync_head.go:140   new network head    {"height": 1970740, "hash": "B3F5C6DA1A7D61517D1C78B473B4587BFD51122257921DF2382C9D31E70816B9"}
2024-07-25T14:35:35.452-0400    INFO    das das/worker.go:161   sampled header  {"type": "recent", "height": 1970740, "hash": "B3F5C6DA1A7D61517D1C78B473B4587BFD51122257921DF2382C9D31E70816B9", "square width": 32, "data root": "3709B2DF3CA20B3848FD1520C3474274392D4BC558AEB27AE9BCF67BE4B34439", "finished (s)": 0.797861833}
2024-07-25T14:35:35.585-0400    INFO    header/store    store/store.go:367  new head    "height": 2561, "hash": "F829A24D3700B8321F21B13232AA2539FEF157E894A395BCAE782D997E3FA523"}
2024-07-25T14:35:35.586-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2002, "to": 2101, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 23.946027583}
2024-07-25T14:35:35.586-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2202, "to": 2301, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 17.288435208}
2024-07-25T14:35:35.586-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2102, "to": 2201, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 17.28847375}
2024-07-25T14:35:35.586-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2402, "to": 2501, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 17.288464166}
2024-07-25T14:35:35.586-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2302, "to": 2401, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 17.28848575}
2024-07-25T14:35:41.020-0400    INFO    header/store    store/store.go:367  new head    "height": 3073, "hash": "8E14DC13AB8B3A7ED45C17ADD3214B6D49E11CE268F1BE234A30A09E37B46455"}
2024-07-25T14:35:41.021-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2502, "to": 2601, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 22.72330075}
2024-07-25T14:35:41.021-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2802, "to": 2901, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 16.653290667}
2024-07-25T14:35:41.021-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2602, "to": 2701, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 16.653656875}
2024-07-25T14:35:41.021-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2702, "to": 2801, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 16.653342084}
2024-07-25T14:35:41.021-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 2902, "to": 3001, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 16.653285333}
2024-07-25T14:35:45.407-0400    INFO    header/store    store/store.go:367  new head    "height": 3585, "hash": "40419F7A0D4E9EEFD8510EF99FBD6F32E48669044D252AFAD1FC29DD20449FF2"}
2024-07-25T14:35:45.408-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3002, "to": 3101, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 21.039979792}
2024-07-25T14:35:45.408-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3402, "to": 3501, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 15.325489625}
2024-07-25T14:35:45.408-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3302, "to": 3401, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 15.325597208}
2024-07-25T14:35:45.408-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3102, "to": 3201, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 15.325732208}
2024-07-25T14:35:45.408-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3202, "to": 3301, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 15.325674541}
2024-07-25T14:35:46.717-0400    INFO    header/sync sync/sync_head.go:140   new network head    {"height": 1970741, "hash": "FB012BB793C5675C22B9ABC21C7A12338191950C52A220256BB7042CDED83327"}
2024-07-25T14:35:47.338-0400    INFO    das das/worker.go:161   sampled header  {"type": "recent", "height": 1970741, "hash": "FB012BB793C5675C22B9ABC21C7A12338191950C52A220256BB7042CDED83327", "square width": 32, "data root": "494DCC2071E879347574D0F6EA6A4A9293D669B27F7350814431DC66CFF86A5D", "finished (s)": 0.620483791}
2024-07-25T14:35:51.081-0400    ERROR   state   state/core_access.go:355    balance for account celestia1j6ppzj5e3xkz8d22ys4qyz55ddwtxttu6np72y does not exist at block height 1970740
2024-07-25T14:35:51.974-0400    INFO    header/store    store/store.go:367  new head    "height": 4097, "hash": "7038B7E994CF5FFD4519372079589C9C77F813EEC3F7F6992915FE9086D6284A"}
2024-07-25T14:35:51.974-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3502, "to": 3601, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 21.8917765}
2024-07-25T14:35:51.975-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3802, "to": 3901, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 16.388549}
2024-07-25T14:35:51.975-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3702, "to": 3801, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 16.3886275}
2024-07-25T14:35:51.975-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3902, "to": 4001, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 16.388571208}
2024-07-25T14:35:51.975-0400    INFO    das das/worker.go:94    finished sampling headers   {"type": "catchup", "from": 3602, "to": 3701, "errors": 0, "# of headers skipped as outside of sampling window": 100, "finished (s)": 16.388719541}
2024-07-25T14:35:52.831-0400    ERROR   state   state/core_access.go:355    balance for account celestia1j6ppzj5e3xkz8d22ys4qyz55ddwtxttu6np72y does not exist at block height 1970740
^C2024-07-25T14:35:55.961-0400  INFO    rpc rpc/server.go:108   server stopped
2024-07-25T14:35:55.961-0400    INFO    das das/store.go:60 stored checkpoint to disk: SampleFrom: 5602, NetworkHead: 1970741, Workers: 16
2024-07-25T14:35:55.961-0400    INFO    das das/store.go:60 stored checkpoint to disk: SampleFrom: 5602, NetworkHead: 1970741, Workers: 16
2024-07-25T14:35:55.961-0400    INFO    header/p2p  p2p/server.go:86    server: stopping server
2024-07-25T14:35:55.961-0400    INFO    header/sync sync/sync.go:216    syncing headers {"from": 4098, "to": 1970741}
2024-07-25T14:35:55.965-0400    INFO    pidstore    pidstore/pidstore.go:84 Persisted peers successfully    {"amount": 99}
2024-07-25T14:35:55.984-0400    WARN    peerstore   pstoremem/addr_book.go:248  Was passed p2p address with a different peerId. found: 12D3KooWBkY9UHLUU4kMkijYi3FoavJTEBLcG2XXXwWucR7b1xuE, expected: 12D3KooWSkkB4XWfWwAFpva2g7uSdoHqWtjVT7QyF72Qs6E5RHdm
2024-07-25T14:35:56.009-0400    INFO    badger4 v4@v4.2.1-0.20240106094458-1c417aa3799c/db.go:546   Lifetime L0 stalled for: 0s

2024-07-25T14:35:56.055-0400    INFO    badger4 v4@v4.2.1-0.20240106094458-1c417aa3799c/db.go:625   
Level 0 [ ]: NumTables: 00. Size: 0 B of 0 B. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 16 MiB
Level 1 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Level 2 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Level 3 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Level 4 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Level 5 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Level 6 [B]: NumTables: 01. Size: 798 KiB of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Level Done

Suggested fix

The DA node shouldn't start after it checks consensus RPC and receives a different chain ID. It shoulod error accordingly and fail to start node.

Notes

The log 2024-07-25T14:35:51.081-0400 ERROR state state/core_access.go:355 balance for account celestia1j6ppzj5e3xkz8d22ys4qyz55ddwtxttu6np72y does not exist at block height 1970740 only makes it appear that the account is not funded.

renaynay commented 1 month ago

The way to resolve this is to do a Status check to the core endpoint over RPC and the chain ID would be contained in the response.

(but keep in mind, this endpoint exists only over RPC so it would need to be implemented in gRPC by @celestiaorg/celestia-core as we only communicate via gRPC to core when it's for state-related queries).

renaynay commented 1 month ago

Actually, I stand corrected. There's a gRPC endpoint for it already so should be very easy to implement on Start of core accessor:

https://github.com/celestiaorg/cosmos-sdk/blob/0dc27d4788a6dfe72ae15f049e344f476eef49f2/proto/cosmos/base/tendermint/v1beta1/query.proto#L19

ChainID == Network here (resp.DefaultNodeInfo.Network)

kobakaku commented 1 month ago

How do you think it’s best to retrieve the network type on which the node is running within core_access.go ? In this example, how would you retrieve celestia?

jpserrat commented 1 month ago

I am working on this.

cc: @jcstein, @renaynay