Closed raymondclowe closed 2 years ago
I think this is a duplicate of https://github.com/lightningnetwork/lnd/issues/6241, which should be fixed in the upcoming v0.14.3-beta
(for which the RC is being prepared here https://github.com/lightningnetwork/lnd/pull/6366).
I think this is a duplicate of #6241, which should be fixed in the upcoming
v0.14.3-beta
(for which the RC is being prepared here #6366).
Thanks, certainly sounds similar.
Two points:
No, --reset-wallet-transactions
can't fix this, only the mentioned version can.
The 15455 sats of your channel aren't lost. You just can't currently sweep them because of the bug. You should be able to with v0.14.3-beta
.
Someone else collected the 330 sats that were used as the anchor output. See https://bitcoinops.org/en/topics/anchor-outputs/ for more explanation and why that's "by design" that someone else can sweep them.
Ah, I just looked at the outputs again, and I was wrong. The 15792 sats were already swept to your wallet (4641f2a7a9d18af733f58ce3ffa0f65dc5f5f91510fc27fe4c4468083aca1a1f
). So the only thing that will change with v0.14.3-beta
is that the "ghost UTXO" will likely disappear.
So transaction 4641f2a7a9d18af733f58ce3ffa0f65dc5f5f91510fc27fe4c4468083aca1a1f
sent it to address bc1qupvlpas722umft6rr8zn3a20tm68nwe43xrxqx
, but how is that mine? I don't see it when I do lncli listunspent
.
It was just a guess, since I don't know what your balance of the channel was. Do you see ab20d89bde30580d5faf21de3f2012ef3732525cbf86ac2821dbac398d520406
in your list of unspent?
No, scratch that. I see from your log that your sweeper tries to sweep 4641f2a7a9d18af733f58ce3ffa0f65dc5f5f91510fc27fe4c4468083aca1a1f:0
into the wallet but fails because of the already swept anchor. Not sure why that extra transaction is necessary. So you do need to wait for lnd v0.14.3-beta
it seems.
Ok thanks, I'll wait.
Appreciate you taking the time to check it out. :-)
Closing as this has been addressed by https://github.com/lightningnetwork/lnd/pull/6274. Feel free to re-open if lnd v0.14.3-beta
doesn't fix this.
Closing as this has been addressed by #6274. Feel free to re-open if
lnd v0.14.3-beta
doesn't fix this.
I waited until 0.15 was out and installed it yesterday.
The problem is still there.
Is there something I need to do to make it notice the "unconfirmed" amount?
Hmm, this seems to be happening to other users as well.
Since you upgraded to 0.15 already and that didn't fix it, we need more information to dig deeper.
Could you please set --debuglevel=debug
and --reset-wallet-transactions
on your node (again) and send us the full log since that restart? You can DM me the logs on Slack or send me an email (in my profile).
Also the full output of lncli listunspent
and lncli listchaintxns
would be very useful. Thanks!
Ok I'm running --debuglevel=debug and --reset-wallet-transactions now.
How long do I run it before sending you the logs?
I'd say until the wallet is fully synced (watch the log for progress) and then maybe two hours more to make sure any rescans finish as well?
What do I look for in the log to tell the wallet is fully synced?
I grepped for "wallet" and got:
2022-07-25 15:11:19.410 [DBG] WLKT: WalletKit RPC server successfully registered with root gRPC server
2022-07-25 15:11:19.413 [DBG] WLKT: WalletKit REST server successfully registered with root REST server
2022-07-25 15:11:43.110 [INF] LTND: Attempting automatic wallet unlock with password provided in file
2022-07-25 15:11:44.955 [INF] LNWL: Opened wallet
2022-07-25 15:11:44.955 [INF] LNWL: Dropping btcwallet transaction history
2022-07-25 15:11:46.745 [INF] LNWL: Opened wallet
2022-07-25 15:11:48.791 [INF] LNWL: The wallet has been unlocked without a time limit
2022-07-25 15:11:48.798 [INF] CHRE: LightningWallet opened
2022-07-25 15:11:49.003 [DBG] LTND: Populating dependencies for sub RPC server: WalletKitRPC
2022-07-25 15:11:49.014 [DBG] RPCS: Starting sub RPC server: WalletKitRPC
Grepped for sync and got:
2022-07-25 15:11:19.422 [INF] LTND: Opening bbolt database, sync_freelist=false, auto_compact=true
2022-07-25 15:11:44.187 [INF] BTCN: Waiting for more block headers, then will start cfheaders sync from height 746434...
2022-07-25 15:11:44.188 [INF] BTCN: Starting cfheaders sync from (block_height=746434, block_hash=00000000000000000006cfb7ec409bd6f810840da242253884b6f6612ba2a9e3) to (block_height=746434, block_hash=00000000000000000006cfb7ec409bd6f810840da242253884b6f6612ba2a9e3)
2022-07-25 15:11:44.188 [INF] BTCN: Starting cfheaders sync for filter_type=0
2022-07-25 15:11:48.799 [DBG] LNWL: Waiting for chain backend to sync to tip
2022-07-25 15:11:49.019 [INF] LTND: Waiting for chain backend to finish sync, start_height=746434
2022-07-25 15:11:49.815 [DBG] LNWL: Chain backend synced to tip!
2022-07-26 05:45:23.003 [WRN] BTCN: No sync peer candidates available
2022-07-26 05:48:28.345 [WRN] BTCN: No sync peer candidates available
I saw the "Starting rescan" so grepped for "rescan" and got:
2022-07-25 15:11:49.820 [DBG] BTCN: Starting rescan from known block 746434 (00000000000000000006cfb7ec409bd6f810840da242253884b6f6612ba2a9e3)
2022-07-25 15:11:49.870 [INF] LNWL: Started rescan from block 0000000000000000000066f2e0a0937b73c26a10d33fbf6d9d0811ebb4866dd9 (height 695311) for 30 addresses
2022-07-25 15:11:49.870 [DBG] BTCN: Starting rescan from known block 695311 (0000000000000000000066f2e0a0937b73c26a10d33fbf6d9d0811ebb4866dd9)
At the moment it is just doing ping and pong with an occasional addrv2.
Can you check if lncli getinfo
returns synced_to_chain: true
and lncli getrecoveryinfo
returns recovery_finished: true
?
If both is the case, then you can send me the logs.
No, they both return false, synced_to_graph also false. And both channels (I only had 2) are inactive. Feel this isn't working so I'm restarting everything, cold boot.
Now that I know to look for "getrecoveryinfo" I'll try again and see if it turns true at some point.
One other point; this node is using Neutrino backend which I gather is unusual. At the time I set it up I didn't have a working bitcoin node, but I do now. I have on another machine on the LAN a bitcoind archival with txindex. If it would be helpful for troubleshooting I could try to switch the lnd backend to point to the bitcoind. In theory at least! :-)
Switching from neutrino to bitcoind should work. But the other way won't be possible.
In that case I'll try that later and try --reset-wallet-transactions
again first.
Is getrecoveryinfo
supposed to say something is in progress?
All I get is:
{
"recovery_mode": false,
"recovery_finished": false,
"progress": 0
}
Still the same 5 hours later.
Ah, I think I remembered incorrectly and recovery mode is only active when restoring from seed, sorry. I think all you have to go by is the log. This is how it looks on my dev machine for example:
2022-07-29 11:25:31.138 [INF] LTND: Waiting for chain backend to finish sync, start_height=4856
2022-07-29 11:25:31.146 [INF] LNWL: Started rescan from block 72791a8901191a2140743bac67091a8a9dfb90029ff6fdda4f1009e45240787e (height 319) for 1 address
2022-07-29 11:25:31.150 [INF] LNWL: Catching up block hashes to height 320, this might take a while
2022-07-29 11:25:31.151 [DBG] LNWL: Filtering block 320 (3fdb7f2aa72a5f87127be622530ee010511aed4933f45a9d5879f34ed77aebb4) with 1 transactions
2022-07-29 11:25:31.153 [DBG] LNWL: Filtering block 321 (350ef9fe548089b0a5dcbd8cb826884076642836169d7e1ae9b3680212009ade) with 1 transactions
2022-07-29 11:25:31.153 [INF] LNWL: Done catching up block hashes
2022-07-29 11:25:31.153 [INF] LNWL: Rescanned through block 72791a8901191a2140743bac67091a8a9dfb90029ff6fdda4f1009e45240787e (height 320)
2022-07-29 11:25:31.154 [DBG] LNWL: Filtering block 322 (35c5a731e3dbbb4e9cb915d26932127f58c3e74a7d37a619a973dd773c9d7165) with 1 transactions
2022-07-29 11:25:31.155 [DBG] LNWL: Filtering block 323 (1dd38ba9b3717e926132d89a026acaf50ed992f84020c9208151cd5c46baf358) with 1 transactions
2022-07-29 11:25:31.156 [DBG] LNWL: Filtering block 324 (24749a64c11262c61633df13ea159fe07f3ee8008c71736fed8a327334381d77) with 1 transactions
2022-07-29 11:25:31.157 [DBG] LNWL: Filtering block 325 (4e81e0ddd148e00680f9b9e963016a8e4597bf4a59b0a1c52de815be1e6f6ea9) with 1 transactions
...
2022-07-29 11:25:35.017 [DBG] LNWL: Filtering block 4856 (010398a6f381297c5d5a0b775c4b745ef184dea81e96f7a3dca4e04ea271ff2c) with 1 transactions
2022-07-29 11:25:35.018 [INF] LNWL: Catching up block hashes to height 4856, this might take a while
2022-07-29 11:25:35.019 [INF] LNWL: Done catching up block hashes
2022-07-29 11:25:35.019 [INF] LNWL: Finished rescan for 1 address (synced to block 010398a6f381297c5d5a0b775c4b745ef184dea81e96f7a3dca4e04ea271ff2c, height 4856)
2022-07-29 11:25:35.148 [INF] LTND: Chain backend is fully synced (end_height=4856)!
Once you see Chain backend is fully synced
you're good to go.
It ran over the weekend with --debuglevel=debug --reset-wallet-transactions
and it still isn't synced.
"synced_to_chain": false,
"synced_to_graph": false,
The only references to sync in the log are
2022-07-29 20:20:32.248 [WRN] BTCN: No sync peer candidates available
2022-07-30 03:06:53.785 [WRN] BTCN: No sync peer candidates available
2022-07-30 04:34:06.042 [WRN] BTCN: No sync peer candidates available
2022-07-30 05:45:26.235 [WRN] BTCN: No sync peer candidates available
2022-07-30 05:48:05.820 [WRN] BTCN: No sync peer candidates available
2022-07-31 05:45:29.722 [WRN] BTCN: No sync peer candidates available
2022-08-01 01:00:31.550 [WRN] BTCN: No sync peer candidates available
2022-08-01 05:45:23.544 [WRN] BTCN: No sync peer candidates available
I also saw
2022-07-29 11:30:12.068 [ERR] LNWL: Neutrino rescan ended with error: couldn't retrieve block 0000000000000000000774078e38fb6978e113b68b97ba9bf80fa0f7bf95ae77 from network
I guess I have to switch to the bitcoind backend, but I'm afraid I may not be able to make anything work again so I'll try to get what funds I have in it out first, which may be difficult so will have to wait.
I'll update when I've done that.
It looks like the Neutrino peer you were using isn't reliably serving you blocks anymore. You can also try with another peer first.
@raymondclowe what about that update ? I get the same "Neutrino rescan ended with error..." did switching to bitcoind backend helped?
I'm sorry I don't recall the details. I have given up trying to run lnd.
Background
After the Telegram lightning watch bot went down a few days ago the channel to it was force closed but the funds in the channel have not reappeared for me. There is a transaction listed but it is not found in mempool.space. There is another transaction with approx the right amount, but the money isn't found anywhere, it has gone to another address I can't see I think; not very clear on this last point.
This node hasn't been touched in several months, apart from upgrading lnd and restarting, as it is my test setup for "I'll get around to learning more about this when I have time" which of course never happens.
When I first started to troubleshoot this there were 3 utxos in this strange 0-confirmations state so I took the advice to use
--reset-wallet-transactions
and after an hour the situation improved and now has only one like this. I ran it again to see if it would clear up any more but has not changed again.Your environment
The problem
Two-fold issue from my perspective
Transaction with no confirmations
Is this normal? lnd can see this utxo but the transaction doesn't exist and hence no balance in the address.
Transaction with funds gone somewhere I can't see
Separately, in RTL there is a listing for the force closed transaction. I don't know how to see this via the lncli.
In that listing it shows the closing transaction as being
10034c258395cb0211ec0edd066d11cb1b5a0cec0aaa0890f947959b88f0b409
It shows the sats moving to an address, and then some time later moving to another one. I can't see this as an available balance unless it is the unconfirmed amount described above, though the amount doesn't quite match.
Expected behaviour
I would expect the funds from the force closed to become available to me.
Actual behaviour
It didn't
Log grep for the transaction id s