Closed NCrashed closed 5 months ago
kaloudis
commented
5 months ago @NCrashed please upgrade to ZEUS v0.8.2
If channels remain offline, try these steps https://docs.zeusln.app/for-users/embedded-node/faq/#my-channels-are-showing-as-offline-how-can-i-get-them-to-show-as-active-again
NCrashed
commented
5 months ago I updated to 0.8.2. Next, I resynced neutrino filters. Finally, I initiated graphic resync. The issue persits without any updates in its behavior.
New logs:
18:55:04.356 [INF] BTCN: Starting to query for cfheaders from checkpoint_interval=826, checkpoints=828
18:55:04.356 [INF] BTCN: Attempting to query for 1 cfheader batches
18:55:05.371 [INF] BTCN: Successfully got filter headers for 828 checkpoints
18:55:05.372 [INF] BTCN: Waiting for more block headers, then will start cfheaders sync from height 828000...
18:55:05.372 [INF] BTCN: Starting cfheaders sync from (block_height=828000, block_hash=0000000000000000000019dd80d72b4eceb4ae189b2b991d4dec93c5bb39bc53) to (block_height=832000, block_hash=000000000000000000009f9aa57e72b0fc1ca64c6851069bc1c28f0b212a1bf5)
18:55:05.372 [INF] BTCN: Starting cfheaders sync for filter_type=0
18:55:05.766 [INF] BTCN: Fetching set of checkpointed cfheaders filters from height=828000, hash=1f793f807ab2a5c6ee4736707c53daa5a6f9350d080fcd32a834bd24a8066484
18:55:05.767 [INF] BTCN: Starting to query for cfheaders from checkpoint_interval=828, checkpoints=832
18:55:05.767 [INF] BTCN: Attempting to query for 2 cfheader batches
18:55:06.142 [INF] BTCN: Verified 10000 filter headers in the last 10.34s (height 828001, 2024-01-29 19:48:43 +0000 UTC)
18:55:07.768 [WRN] LNWL: Query(424) from peer 45.79.41.132:8333 failed, rescheduling: did not get response before timeout
18:55:08.851 [INF] BTCN: Successfully got filter headers for 832 checkpoints
18:55:08.852 [INF] BTCN: Waiting for more block headers, then will start cfheaders sync from height 832000...
18:55:08.852 [INF] BTCN: Starting cfheaders sync from (block_height=832000, block_hash=000000000000000000009f9aa57e72b0fc1ca64c6851069bc1c28f0b212a1bf5) to (block_height=834000, block_hash=00000000000000000003193990cb2a085cfe0491a6db5b3b9ce564e8b9644999)
18:55:08.852 [INF] BTCN: Starting cfheaders sync for filter_type=0
18:55:09.892 [INF] BTCN: Fetching set of checkpointed cfheaders filters from height=832000, hash=7453cdb2b8196eddbc8ede0b4bc92de5edfc9d0cceed6d4f78b1a9f691740ed1
18:55:09.892 [INF] BTCN: Starting to query for cfheaders from checkpoint_interval=832, checkpoints=834
18:55:09.892 [INF] BTCN: Attempting to query for 1 cfheader batches
18:55:09.901 [INF] BTCN: Processed 12000 blocks in the last 11.94s (height 834001, 2024-03-10 06:10:12 +0000 UTC)
18:55:10.940 [INF] BTCN: Successfully got filter headers for 834 checkpoints
18:55:10.940 [INF] BTCN: Waiting for more block headers, then will start cfheaders sync from height 834000...
18:55:10.940 [INF] BTCN: Starting cfheaders sync from (block_height=834000, block_hash=00000000000000000003193990cb2a085cfe0491a6db5b3b9ce564e8b9644999) to (block_height=835259, block_hash=00000000000000000003053b0a32837fbd485e6ec3d8e84218e6ab3e2d54fe90)
18:55:10.941 [INF] BTCN: Starting cfheaders sync for filter_type=0
18:55:11.240 [INF] BTCN: Fetching set of checkpointed cfheaders filters from height=834000, hash=23b1edb2a84782863a36f7f772c9217b451d73b30f7d4ab14e902305a39b9732
18:55:11.241 [INF] BTCN: Starting to query for cfheaders from checkpoint_interval=834, checkpoints=835
18:55:11.241 [INF] BTCN: Attempting to query for 1 cfheader batches
18:55:11.577 [INF] BTCN: Successfully got filter headers for 835 checkpoints
18:55:11.578 [INF] BTCN: Fully caught up with cfheaders at height 835259, waiting at tip for new blocks
18:55:11.578 [INF] BTCN: Attempting to fetch set of un-checkpointed filters at height=835259, hash=00000000000000000003053b0a32837fbd485e6ec3d8e84218e6ab3e2d54fe90
18:55:12.741 [INF] BTCN: Registering block subscription: id=2
18:55:12.849 [INF] LNWL: Started rescan from block 000000000000000000001be3e5929be506d4e2c8cd1b7ba4fa9422032434368f (height 819112) for 6 addresses
18:55:12.850 [INF] BTCN: Canceling block subscription: id=2
18:55:14.854 [WRN] LNWL: Query(427) from peer 45.79.195.29:8333 failed, rescheduling: did not get response before timeout
18:55:18.856 [WRN] LNWL: Query(427) from peer 45.79.195.29:8333 failed and reached maximum number of retries, not rescheduling: did not get response before timeout
18:55:20.279 [ERR] RPCS: [/lnrpc.Lightning/ConnectPeer]: server is still in the process of starting
18:55:51.369 [INF] LTND: Received shutdown request.
18:55:51.369 [INF] LTND: Shutting down...
18:55:51.369 [INF] LTND: Gracefully shutting down.
18:55:59.965 [INF] LTND: Listening on the p2p interface is disabled!
18:55:59.966 [INF] LTND: Version: 0.17.4-beta commit=v0.17.4-beta-zeus.1, build=production, logging=default, debuglevel=info
18:55:59.966 [INF] LTND: Active chain: Bitcoin (network=mainnet)
18:55:59.968 [INF] RPCS: RPC server listening on bufconn
18:55:59.978 [INF] RPCS: gRPC proxy started at 127.0.0.1:8080
18:55:59.978 [INF] LTND: Opening the main database, this might take a few minutes...
18:55:59.978 [INF] LTND: Opening bbolt database, sync_freelist=true, auto_compact=false
18:55:59.996 [INF] LTND: Creating local graph and channel state DB instances
18:56:00.881 [INF] CHDB: Checking for schema update: latest_version=31, db_version=31
18:56:00.881 [INF] CHDB: Checking for optional update: prune_revocation_log=false, db_version=empty
18:56:00.881 [INF] LTND: Database(s) now open (time_to_open=902.595833ms)!
18:56:00.913 [INF] BTCN: Loaded 2051 addresses from file '/data/user/0/app.zeusln.zeus/files/data/chain/bitcoin/mainnet/peers.json'
18:56:00.913 [INF] BTCN: Registering block subscription: id=1
18:56:00.913 [INF] LTND: We're not running within systemd or the service type is not 'notify'
18:56:00.913 [INF] LTND: Waiting for wallet encryption password. Use `lncli create` to create a wallet, `lncli unlock` to unlock an existing wallet, or `lncli changepassword` to change the password of an existing wallet and unlock it.
18:56:00.913 [INF] BTCN: Broadcaster now active
18:56:00.918 [INF] CMGR: 25 addresses found from DNS seed x49.seed.bitcoin.sipa.be
18:56:00.922 [INF] CMGR: 36 addresses found from DNS seed dnsseed.bitcoin.dashjr.org
18:56:00.922 [INF] CMGR: 36 addresses found from DNS seed x49.seed.btc.petertodd.net
18:56:00.924 [INF] CMGR: 36 addresses found from DNS seed x49.seed.bitcoin.jonasschnelli.ch
18:56:01.137 [INF] CMGR: 21 addresses found from DNS seed x49.dnsseed.bluematt.me
18:56:01.153 [INF] CMGR: 37 addresses found from DNS seed x49.seed.bitcoinstats.com
18:56:01.156 [INF] CMGR: 40 addresses found from DNS seed seed.bitnodes.io
18:56:01.454 [INF] BTCN: Waiting for more block headers, then will start cfheaders sync from height 835259...
18:56:01.454 [INF] BTCN: New valid peer 45.79.195.29:8333 (outbound) (/btcwire:0.5.0/btcd:0.23.3/)
18:56:01.455 [INF] BTCN: Starting cfheaders sync from (block_height=835259, block_hash=00000000000000000003053b0a32837fbd485e6ec3d8e84218e6ab3e2d54fe90) to (block_height=835259, block_hash=00000000000000000003053b0a32837fbd485e6ec3d8e84218e6ab3e2d54fe90)
18:56:01.455 [INF] BTCN: Starting cfheaders sync for filter_type=0
18:56:01.455 [INF] BTCN: Syncing to block height 835259 from peer 45.79.195.29:8333
18:56:01.455 [INF] BTCN: Fetching set of headers from tip (height=835259) from peer 45.79.195.29:8333
18:56:01.552 [INF] BTCN: New valid peer 45.79.41.132:8333 (outbound) (/btcwire:0.5.0/btcd:0.23.3/)
18:56:02.006 [INF] BTCN: Fetching set of checkpointed cfheaders filters from height=835259, hash=babc72dd5b217dbe6cdace96bd8aebd3d025421fac497dc6a0beba6d5830457b
18:56:02.006 [INF] BTCN: Starting to query for cfheaders from checkpoint_interval=835, checkpoints=835
18:56:02.007 [INF] BTCN: Fully caught up with cfheaders at height 835259, waiting at tip for new blocks
18:56:02.154 [INF] LNWL: Opened wallet
18:56:02.303 [INF] CHRE: Primary chain is set to: bitcoin
18:56:02.333 [INF] CHRE: Using external fee estimator https://nodes.lightning.computer/fees/v1/btc-fee-estimates.json: cached=true
18:56:02.333 [INF] LNWL: Starting web API fee estimator
18:56:03.780 [INF] LNWL: The wallet has been unlocked without a time limit
18:56:03.782 [INF] CHRE: LightningWallet opened
18:56:03.809 [INF] HSWC: Cleaning circuits from disk for closed channels
18:56:03.809 [INF] HSWC: Finished cleaning: no closed channels found, no actions taken.
18:56:03.809 [INF] HSWC: Restoring in-memory circuit state from disk
18:56:03.818 [INF] HSWC: Payment circuits loaded: num_pending=0, num_open=0
18:56:03.822 [INF] HSWC: Trimming open circuits for chan_id=16000000:0:0, start_htlc_id=9
18:56:03.822 [INF] HSWC: Trimming open circuits for chan_id=813540:941:0, start_htlc_id=10
18:56:03.871 [INF] LTND: Channel backup proxy channel notifier starting
18:56:03.871 [INF] ATPL: Instantiating autopilot with active=false, max_channels=5, allocation=1.000000, min_chan_size=20000, max_chan_size=16777215, private=true, min_confs=1, conf_target=3
18:56:03.872 [INF] LTND: We're not running within systemd or the service type is not 'notify'
18:56:03.872 [INF] LTND: Waiting for chain backend to finish sync, start_height=835259
18:56:04.787 [INF] BTCN: Registering block subscription: id=2
18:56:04.859 [INF] LNWL: Started rescan from block 000000000000000000001be3e5929be506d4e2c8cd1b7ba4fa9422032434368f (height 819112) for 6 addresses
18:56:04.859 [INF] BTCN: Canceling block subscription: id=2
18:56:10.619 [ERR] RPCS: [/lnrpc.Lightning/ConnectPeer]: server is still in the process of starting
18:56:20.191 [INF] LNWL: Catching up block hashes to height 819113, this might take a while
18:56:20.197 [INF] LNWL: Done catching up block hashes
18:56:20.197 [INF] LNWL: Rescanned through block 00000000000000000003c5e909b6b36f6096d6a4fb969c0828d4f7b49c338d00 (height 819113)
18:56:24.265 [WRN] LNWL: Query(1) from peer 45.79.195.29:8333 failed, rescheduling: did not get response before timeout
18:56:41.581 [WRN] LNWL: Query(2) from peer 45.79.195.29:8333 failed, rescheduling: did not get response before timeout
18:56:50.720 [WRN] LNWL: Query(3) from peer 45.79.195.29:8333 failed, rescheduling: did not get response before timeout
18:57:00.374 [ERR] RPCS: [/lnrpc.Lightning/ConnectPeer]: server is still in the process of starting
18:57:09.677 [WRN] LNWL: Query(4) from peer 45.79.195.29:8333 failed, rescheduling: did not get response before timeout
18:57:19.406 [WRN] LNWL: Query(5) from peer 45.79.195.29:8333 failed, rescheduling: did not get response before timeout@NCrashed LND is rescanning as evidenced by the following line:
18:56:20.197 [INF] LNWL: Rescanned through block 00000000000000000003c5e909b6b36f6096d6a4fb969c0828d4f7b49c338d00 (height 819113)
But it seems like you're having issues connecting to the Neutrino peers. As evidenced by the following line:
18:57:09.677 [WRN] LNWL: Query(4) from peer 45.79.195.29:8333 failed, rescheduling: did not get response before timeout
From here you can try this: go to Settings > Embedded node > Advanced > Peers > Neutrino and undo the toggle at the top of the page. Then restart and follow the LND logs for progress.
If that doesn't work we recommend trying on a lower latency connection. Neutrino starts to have issues w/ a 100ms+ ping and becomes unusable around 200ms ping.
NCrashed
commented
5 months ago That toggle apparently didnt help. I think it is really an issue with my latency. I am in South Asia now and any European server is 250+ ms away from me.
NCrashed
commented
5 months ago So, the only option left is force close?
kaloudis
commented
5 months ago You could try to find a node in your region with block filters enabled https://bitnodes.io/nodes/?q=NODE_COMPACT_FILTERS
NCrashed
commented
5 months ago Oh, that was a very good advice. I found a node in Kuala Lumpur and all is back online. I should consider deploying own node with filters at my place. I am sure that I will be not the only person with such issue there.
Describe the bug
I haven't opened the app for a while now, perhaps, a month. Now every time I start Zeus all channels are offline and flushing every 5 secs like trying to reconnect.
Peers are clearnet, VPN to other country doesnt affect the issue.
Version: 0.8.0-beta1
Peers:
Reproduce
Reproduce the bug step by step.
ZEUS version
v0.8.0
Node interface
Embedded LND
Network
Clearnet
Device
Huawei p30 pro
Device operating system
Android 10
Log output