pendingchannels rpc gives wrong htlc stage and sweeps in 2 separate transactions after turning on lnd

antonilol commented 2 years ago

Background

Describe your issue here.

i made a channel 800k sats and keysended 50k sats to the other side. there were no pending HTLCs at this point in time and remote balance was 50k sats. then, i made 2 HTLCs of 50k sats each (with a swap), turned off my node and gave the remote party the preimage of 1 of the 2 swaps (by redeeming on-chain). the other party will try to fulfill the HTLC but i am not online, so force closed the channel before it times out to claim it on chain.

when i turned the node back on, it rescanned the chain and obviously saw the force close, it directly swept the local output (:3), it took some time before it swept the timed out HTLC (:2) and in between a block was mined (dont know it this matters though)

before the timed out HTLC was swept the pendingchannels rpc said it was on stage 2 (after the sweep confirmed it wasnt pending anymore)

{
    "total_limbo_balance": "749730",
    "pending_open_channels": [
    ],
    "pending_closing_channels": [
    ],
    "pending_force_closing_channels": [
        {
            "channel": {
                "remote_node_pub": "03f060953bef5b777dc77e44afa3859d022fc1a77c55138deb232ad7255e869c00",
                "channel_point": "23e8ac05ab5e50977e65224efaf66a228bd2115814afafb86bf22c84d526e022:1",
                "capacity": "800000",
                "local_balance": "649730",
                "remote_balance": "0",
                "local_chan_reserve_sat": "0",
                "remote_chan_reserve_sat": "0",
                "initiator": "INITIATOR_LOCAL",
                "commitment_type": "STATIC_REMOTE_KEY",
                "num_forwarding_packages": "0",
                "chan_status_flags": ""
            },
            "closing_txid": "de55d4e3b8c8d541b85cc51666f19b5acedb8a31dd308e9bc6b3ee1e899e028f",
            "limbo_balance": "749730",
            "maturity_height": 2196616,
            "blocks_til_maturity": -46,
            "recovered_balance": "0",
            "pending_htlcs": [
                {
                    "incoming": false,
                    "amount": "50000",
                    "outpoint": "de55d4e3b8c8d541b85cc51666f19b5acedb8a31dd308e9bc6b3ee1e899e028f:1",
                    "maturity_height": 2196625,
                    "blocks_til_maturity": -37,
                    "stage": 2
                },
                {
                    "incoming": false,
                    "amount": "50000",
                    "outpoint": "de55d4e3b8c8d541b85cc51666f19b5acedb8a31dd308e9bc6b3ee1e899e028f:2",
                    "maturity_height": 2196625,
                    "blocks_til_maturity": -37,
                    "stage": 2      <-- this
                }
            ],
            "anchor": "LIMBO"
        }
    ],
    "waiting_close_channels": [
    ]
}

after a block, the fulfilled HTLC (:1) had stage 0 for some reason, nothing happened to that output in this block, also the local sweep got mined, the timed out HTLC sweep wasnt made yet, note the recovered balance == the local balance

{
    "total_limbo_balance": "50000",
    "pending_open_channels": [
    ],
    "pending_closing_channels": [
    ],
    "pending_force_closing_channels": [
        {
            "channel": {
                "remote_node_pub": "03f060953bef5b777dc77e44afa3859d022fc1a77c55138deb232ad7255e869c00",
                "channel_point": "23e8ac05ab5e50977e65224efaf66a228bd2115814afafb86bf22c84d526e022:1",
                "capacity": "800000",
                "local_balance": "649730",
                "remote_balance": "0",
                "local_chan_reserve_sat": "0",
                "remote_chan_reserve_sat": "0",
                "initiator": "INITIATOR_LOCAL",
                "commitment_type": "STATIC_REMOTE_KEY",
                "num_forwarding_packages": "0",
                "chan_status_flags": ""
            },
            "closing_txid": "de55d4e3b8c8d541b85cc51666f19b5acedb8a31dd308e9bc6b3ee1e899e028f",
            "limbo_balance": "50000",
            "maturity_height": 2196616,
            "blocks_til_maturity": -47,
            "recovered_balance": "699730",
            "pending_htlcs": [
                {
                    "incoming": false,
                    "amount": "50000",
                    "outpoint": "de55d4e3b8c8d541b85cc51666f19b5acedb8a31dd308e9bc6b3ee1e899e028f:1",
                    "maturity_height": 2196616,
                    "blocks_til_maturity": -47,
                    "stage": 0
                },
                {
                    "incoming": false,
                    "amount": "50000",
                    "outpoint": "de55d4e3b8c8d541b85cc51666f19b5acedb8a31dd308e9bc6b3ee1e899e028f:2",
                    "maturity_height": 2196625,
                    "blocks_til_maturity": -38,
                    "stage": 2
                }
            ],
            "anchor": "LIMBO"
        }
    ],
    "waiting_close_channels": [
    ]
}

another block was mined with the timed out HTLC sweep, and the channel was fully closed on my side

Your environment

version of lnd lnd version 0.14.2-beta commit=v0.14.2-beta
which operating system (uname -a on *Nix) Linux node 5.16.14-arch1-1 #1 SMP PREEMPT Fri, 11 Mar 2022 17:40:36 +0000 x86_64 GNU/Linux
version of btcd, bitcoind, or other backend Bitcoin Core version v22.0.0
any other relevant environment details on testnet

Steps to reproduce

Tell us how to reproduce this issue. Please provide stacktraces and links to code in question.

what i did in 'Background', for the swap i used https://testnet.boltz.exchange/

Expected behaviour

Tell us what should happen

correct stage and blocks_til_maturity
one sweep tx

Actual behaviour

Tell us what happens instead

the strange this with the stage and blocks_til_maturity of de55d4e3b8c8d541b85cc51666f19b5acedb8a31dd308e9bc6b3ee1e899e028f:1
the local p2wpkh output was swept separately from the timed out HTLC, which were both spendable when i turned on lnd

tvolk131 commented 2 years ago

Also, what's going on with blocks_til_maturity? The first HTLC changed from -37 to -47.

antonilol commented 2 years ago

Also, what's going on with blocks_til_maturity? The first HTLC changed from -37 to -47.

i dont know, it was already spent on chain when i booted up my lnd. maybe it was trying to time it out and found out later it has been spent by the remote party

tvolk131 commented 2 years ago

Weird... I should have some time to dig into this later tonight or tomorrow.

Crypt-iQ commented 2 years ago

This is because the htlc's are sent to the utxonursery which reports that everything is in stage 2. The change in stage is because the utxonursery requires a block confirmation to clean-up its entries (seen in graduateClass). I don't think this will get addressed since it is:

cosmetic
the utxonursery may get deleted (see https://github.com/lightningnetwork/lnd/issues/3688)

also, static-remote-key channels are on the way towards deprecation

lightningnetwork / lnd