search

lightningnetwork / lnd

Lightning Network Daemon ⚡️
MIT License
7.7k stars 2.09k forks source link

[bug]: "lncli closechannel" may sometimes ignore the `--sat_per_vbyte` argument #8309

Closed kingpalethe closed 10 months ago

kingpalethe commented 10 months ago

Background

Earlier today I tried the lncli closechannel command with a --sat_per_vbyte 100 argument.

The closechannel transaction was broadcast with a much lower fee.

The transaction is now stuck. Here is the relevant result from lncli pendingchannels

 {
            "channel": {
                "remote_node_pub": "037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba",
                "channel_point": "3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0",
                "capacity": "500000",
                "local_balance": "5490",
                "remote_balance": "490032",
                "local_chan_reserve_sat": "5000",
                "remote_chan_reserve_sat": "5000",
                "initiator": "INITIATOR_LOCAL",
                "commitment_type": "ANCHORS",
                "num_forwarding_packages": "0",
                "chan_status_flags": "ChanStatusBorked|ChanStatusCommitBroadcasted|ChanStatusCoopBroadcasted|ChanStatusLocalCloseInitiator",
                "private": false,
                "memo": ""
            },
            "limbo_balance": "5490",
            "commitments": {
                "local_txid": "f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296",
                "remote_txid": "c156ea7e0aedbbbc7b71652e7d01979d04c9869e6f074adeb7dbe1cf4fef0f93",
                "remote_pending_txid": "",
                "local_commit_fee_sat": "3817",
                "remote_commit_fee_sat": "3817",
                "remote_pending_commit_fee_sat": "0"
            },
            "closing_txid": "f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296"
        },

You can see the transaction here: https://mempool.space/tx/f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296 You can see that it was broadcast at 13.7 sats/vB

Your environment

LND: v0.17.2-beta BITCOIND: 25.1

Expected behaviour

LND should use the --sat_per_vbyte argument to set the transaction fee

Actual behaviour

A much lower fee was broadcast.

kingpalethe commented 10 months ago

One thing I though to add to this. The --sat_per_vbyte argument works perfectly when I use it with lncli openchannel

yyforyongyu commented 10 months ago

Maybe the feerate was changed during the fee negotiation period when the channel was closing. Could you share some logs after calling closechannel?

Roasbeef commented 10 months ago

Maybe the feerate was changed during the fee negotiation period when the channel was closing. Could you share some logs after calling closechannel?

Yeah the way things work right now is that both sides sort of "negotiate" what the fee rate should be. This can mean that ultimately, you may not get your exact fee rate.

There's a new version of the co-op close protocol in the works that instead will just have either side pay what they can. It also allows resumption, so you can bump the fee via RBF.

kingpalethe commented 10 months ago

@yyforyongyu Here are lines in the logs which reference either 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba or 3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6 ... channel point and node public key:

2023-12-23 00:19:57.207 [DBG] CNCT: Stopping ChannelArbitrator(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0)
2023-12-23 00:19:57.213 [INF] SRVR: Disconnecting from 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@10.8.0.1:39260
2023-12-23 00:19:57.213 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): disconnecting 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@10.8.0.1:39260, reason: server: DisconnectPeer called
2023-12-23 00:19:57.213 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): unable to read message from peer: read next header: read tcp 10.8.0.2:9735->10.8.0.1:39260: use of closed network connection
2023-12-23 00:20:41.261 [INF] CNCT: Re-publishing force close tx(f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296) for channel 3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0
   PreviousOutPoint: (wire.OutPoint) 3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0,
2023-12-23 00:20:41.273 [WRN] CNCT: Channel 3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0 is in state ChanStatusCoopBroadcasted, but no coop closing tx to re-publish...
2023-12-23 00:20:41.291 [INF] NTFN: New spend subscription: spend_id=38, outpoint=3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0, script=0 e5b050489ed8013a4680a3ebcf3b34f4b06321e730720682645f626bdbb99d8d, height_hint=822478
2023-12-23 00:20:41.291 [INF] NTFN: Dispatching historical spend rescan for outpoint=3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0, script=0 e5b050489ed8013a4680a3ebcf3b34f4b06321e730720682645f626bdbb99d8d, start=822478, end=822478
2023-12-23 00:20:41.846 [INF] CNCT: Close observer for ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) active
2023-12-23 00:20:41.977 [INF] CNCT: ChannelArbitrator(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): starting state=StateCommitmentBroadcasted, trigger=chainTrigger, triggerHeight=822478
2023-12-23 00:20:42.005 [INF] CNCT: ChannelArbitrator(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): trigger chainTrigger moving from state StateCommitmentBroadcasted to StateCommitmentBroadcasted
2023-12-23 00:20:42.260 [WRN] CHBU: Replacing disk backup for ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) w/ newer version
2023-12-23 00:20:42.678 [INF] SRVR: Established connection to: 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@34.78.139.195:9735
2023-12-23 00:20:42.678 [INF] SRVR: Finalizing connection to 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@34.78.139.195:9735, inbound=false
2023-12-23 00:20:42.834 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Loading ChannelPoint(f9a9c355b8be49ed990cb5967d8cb5d273e7a90f94f1a821911e022e6a70d4a9:0), isPending=false
2023-12-23 00:20:42.836 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Loading ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0), isPending=false
2023-12-23 00:20:42.836 [WRN] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) has status ChanStatusBorked|ChanStatusCommitBroadcasted|ChanStatusCoopBroadcasted|ChanStatusLocalCloseInitiator, won't start.
2023-12-23 00:20:42.847 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Delivery addr for channel close: bc1p03lyaa5mh0hem0nyczzr8tdg64hhyr8n35k8r9gc4c56udts5qkqcry079
2023-12-23 00:20:42.848 [INF] CHCL: ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): initiating shutdown
2023-12-23 00:20:42.853 [INF] CHCL: ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): sending shutdown message
2023-12-23 00:20:42.854 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Sending 2 channel sync messages to peer after loading active channels
2023-12-23 00:20:42.854 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Negotiated chan series queries
2023-12-23 00:20:42.854 [INF] DISC: Creating new GossipSyncer for peer=037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba
2023-12-23 00:20:42.854 [ERR] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): resend failed: unable to fetch channel sync messages for peer 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@34.78.139.195:9735: unable to find closed channel summary
2023-12-23 00:20:42.854 [INF] CHCL: Ideal fee for closure of ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) is: 38545 sat (max_fee=115635 sat)
2023-12-23 00:20:42.854 [INF] CHCL: ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): shutdown response received, entering fee negotiation
2023-12-23 00:20:42.854 [ERR] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): unable to process close msg: unable to sign new co op close offer: initiator cannot afford proposed coop close fee
2023-12-23 01:14:14.018 [INF] CNCT: Attempting to force close ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0)
2023-12-23 01:30:19.264 [INF] SRVR: Disconnecting from 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@34.78.139.195:9735
2023-12-23 01:30:19.265 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): disconnecting 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@34.78.139.195:9735, reason: server: DisconnectPeer called
2023-12-23 01:30:19.265 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): unable to read message from peer: read next header: read tcp 10.8.0.2:51356->34.78.139.195:9735: use of closed network connection
2023-12-23 01:31:05.342 [INF] CNCT: Re-publishing force close tx(f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296) for channel 3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0
   PreviousOutPoint: (wire.OutPoint) 3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0,
2023-12-23 01:31:05.354 [WRN] CNCT: Channel 3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0 is in state ChanStatusCoopBroadcasted, but no coop closing tx to re-publish...
2023-12-23 01:31:05.374 [INF] NTFN: New spend subscription: spend_id=23, outpoint=3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0, script=0 e5b050489ed8013a4680a3ebcf3b34f4b06321e730720682645f626bdbb99d8d, height_hint=822489
2023-12-23 01:31:05.374 [INF] NTFN: Dispatching historical spend rescan for outpoint=3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0, script=0 e5b050489ed8013a4680a3ebcf3b34f4b06321e730720682645f626bdbb99d8d, start=822489, end=822489
2023-12-23 01:31:05.780 [INF] CNCT: Close observer for ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) active
2023-12-23 01:31:06.102 [INF] CNCT: ChannelArbitrator(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): starting state=StateCommitmentBroadcasted, trigger=chainTrigger, triggerHeight=822489
2023-12-23 01:31:06.125 [INF] CNCT: ChannelArbitrator(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): trigger chainTrigger moving from state StateCommitmentBroadcasted to StateCommitmentBroadcasted
2023-12-23 01:31:06.318 [WRN] CHBU: Replacing disk backup for ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) w/ newer version
2023-12-23 01:31:06.731 [INF] SRVR: Established connection to: 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@34.78.139.195:9735
2023-12-23 01:31:06.731 [INF] SRVR: Finalizing connection to 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@34.78.139.195:9735, inbound=false
2023-12-23 01:31:06.893 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Loading ChannelPoint(f9a9c355b8be49ed990cb5967d8cb5d273e7a90f94f1a821911e022e6a70d4a9:0), isPending=false
2023-12-23 01:31:06.901 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Loading ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0), isPending=false
2023-12-23 01:31:06.901 [WRN] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) has status ChanStatusBorked|ChanStatusCommitBroadcasted|ChanStatusCoopBroadcasted|ChanStatusLocalCloseInitiator, won't start.
2023-12-23 01:31:06.914 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Delivery addr for channel close: bc1pp30xpqznpwaxmj56zhsj3jg99favj5zwk2ksclzhg0dg96dzqp5spw7xpf
2023-12-23 01:31:06.916 [INF] CHCL: ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): initiating shutdown
2023-12-23 01:31:06.923 [INF] CHCL: ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): sending shutdown message
2023-12-23 01:31:06.923 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Sending 2 channel sync messages to peer after loading active channels
2023-12-23 01:31:06.924 [INF] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): Negotiated chan series queries
2023-12-23 01:31:06.924 [INF] DISC: Creating new GossipSyncer for peer=037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba
2023-12-23 01:31:06.924 [ERR] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): resend failed: unable to fetch channel sync messages for peer 037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba@34.78.139.195:9735: unable to find closed channel summary
2023-12-23 01:31:06.924 [INF] CHCL: Ideal fee for closure of ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) is: 38543 sat (max_fee=115629 sat)
2023-12-23 01:31:06.924 [INF] CHCL: ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0): shutdown response received, entering fee negotiation
2023-12-23 01:31:06.924 [ERR] PEER: Peer(037659a0ac8eb3b8d0a720114efc861d3a940382dcfa1403746b4f8f6b2e8810ba): unable to process close msg: unable to sign new co op close offer: initiator cannot afford proposed coop close fee
kingpalethe commented 10 months ago

also:

2023-12-23 01:31:05.342 [WRN] LTND: Error while broadcasting transaction: mempool fee not met: -26: mempool min fee not met, 3818 < 5120
2023-12-23 01:31:05.342 [INF] LNWL: Inserting unconfirmed transaction f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296
2023-12-23 01:31:05.348 [INF] LNWL: f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296: broadcast failed because of: mempool fee not met: -26: mempool min fee not met, 3818 < 5120
2023-12-23 01:31:05.353 [INF] LNWL: Removed invalid transaction: (*wire.MsgTx)(0xc003b6d240)({
 Version: (int32) 2,
 TxIn: ([]*wire.TxIn) (len=1 cap=1) {
  (*wire.TxIn)(0xc003b677a0)({
   PreviousOutPoint: (wire.OutPoint) 3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0,
   SignatureScript: ([]uint8) {
   },
   Witness: (wire.TxWitness) (len=4 cap=4) {
    ([]uint8) {
    },
    ([]uint8) (len=72 cap=72) {
     00000000  30 45 02 21 00 d6 49 90  2f c8 65 a1 c9 58 f2 8d  |0E.!..I./.e..X..|
     00000010  f1 92 36 90 b8 35 f7 7b  bf 26 12 20 c9 af dd f0  |..6..5.{.&. ....|
     00000020  39 57 0d 4c 44 02 20 68  02 cf 42 b5 c0 da 07 f3  |9W.LD. h..B.....|
     00000030  38 77 ce a8 01 36 86 b6  29 3a a9 98 67 de 13 82  |8w...6..):..g...|
     00000040  f0 4f 63 d1 1c cd 6e 01                           |.Oc...n.|
    },
    ([]uint8) (len=72 cap=72) {
     00000000  30 45 02 21 00 bf 06 71  ca 9e 0f 25 79 44 f6 fe  |0E.!...q...%yD..|
     00000010  2f f8 09 86 07 80 f7 3e  f3 16 3e c1 aa 7d 2f ec  |/......>..>..}/.|
     00000020  3f 31 bf b9 7a 02 20 30  ab 42 d5 08 d8 33 ec 12  |?1..z. 0.B...3..|
     00000030  82 5e dd 4e 53 c2 9f f0  c4 87 58 6c 0b 3c c5 1a  |.^.NS.....Xl.<..|
     00000040  72 2f d4 02 c5 db 4f 01                           |r/....O.|
    },
    ([]uint8) (len=71 cap=71) {
     00000000  52 21 02 6a e5 bd ed 12  ac af 9c f1 8f 97 26 1c  |R!.j..........&.|
     00000010  50 bd a8 e1 01 d2 c7 15  a8 f0 f0 6d 68 29 c6 30  |P..........mh).0|
     00000020  2f 30 a4 21 03 58 e0 d6  4c de e9 61 d2 d6 68 ef  |/0.!.X..L..a..h.|
     00000030  9e 33 18 bd f7 a4 d1 ab  18 7b 74 7f ff db 0f ff  |.3.......{t.....|
     00000040  c7 8f 4c fb 2d 52 ae                              |..L.-R.|
    }
   },
   Sequence: (uint32) 2159519733
  })
 },
 TxOut: ([]*wire.TxOut) (len=4 cap=4) {
  (*wire.TxOut)(0xc003b28380)({
   Value: (int64) 330,
   PkScript: ([]uint8) (len=34 cap=34) {
    00000000  00 20 70 41 2c 5a 17 72  22 bf 22 c0 dd 51 0b de  |. pA,Z.r"."..Q..|
    00000010  b2 86 15 da b2 18 a7 56  a0 71 c4 20 8f ac f7 59  |.......V.q. ...Y|
    00000020  a3 8e                                             |..|
   }
  }),
  (*wire.TxOut)(0xc003b283a0)({
   Value: (int64) 330,
   PkScript: ([]uint8) (len=34 cap=34) {
    00000000  00 20 f9 21 fe d2 72 d2  1b cf 91 c1 0d 65 bf 0b  |. .!..r......e..|
    00000010  9f 0d 95 e0 26 2f 76 59  ec 96 27 87 83 fc b5 c1  |....&/vY..'.....|
    00000020  ca 05                                             |..|
   }
  }),
  (*wire.TxOut)(0xc003b283c0)({
   Value: (int64) 5490,
   PkScript: ([]uint8) (len=34 cap=34) {
    00000000  00 20 8d 7f 03 68 93 3e  0f ff 0b c0 49 39 38 62  |. ...h.>....I98b|
    00000010  64 ba 34 79 95 c8 c1 46  15 d7 18 b8 de 93 ec 7d  |d.4y...F.......}|
    00000020  70 59                                             |pY|
   }
  }),
  (*wire.TxOut)(0xc003b283e0)({
   Value: (int64) 490032,
   PkScript: ([]uint8) (len=34 cap=34) {
    00000000  00 20 42 e8 de 64 4c 71  cc 20 45 bc be 8f 97 be  |. B..dLq. E.....|
    00000010  62 6a 2c 39 02 97 9f 75  6e 77 a6 ce 68 39 b4 45  |bj,9...unw..h9.E|
    00000020  ab ff                                             |..|
   }
  })
 },
 LockTime: (uint32) 546432947
})

2023-12-23 01:31:05.353 [INF] LNWL: Removed invalid transaction: 02000000000101b6ce4b068391ec09b71159cd81aeaf11c80fe266c4f068289dc30a9284e5483c0000000000f5a7b780044a0100000000000022002070412c5a177222bf22c0dd510bdeb28615dab218a756a071c4208facf759a38e4a01000000000000220020f921fed272d21bcf91c10d65bf0b9f0d95e0262f7659ec96278783fcb5c1ca0572150000000000002200208d7f0368933e0fff0bc04939386264ba347995c8c14615d718b8de93ec7d7059307a07000000000022002042e8de644c71cc2045bcbe8f97be626a2c3902979f756e77a6ce6839b445abff0400483045022100d649902fc865a1c958f28df1923690b835f77bbf261220c9afddf039570d4c4402206802cf42b5c0da07f33877cea8013686b6293aa99867de1382f04f63d11ccd6e01483045022100bf0671ca9e0f257944f6fe2ff809860780f73ef3163ec1aa7d2fec3f31bfb97a022030ab42d508d833ec12825edd4e53c29ff0c487586c0b3cc51a722fd402c5db4f01475221026ae5bded12acaf9cf18f97261c50bda8e101d2c715a8f0f06d6829c6302f30a4210358e0d64cdee961d2d668ef9e3318bdf7a4d1ab187b747fffdb0fffc78f4cfb2d52aeb3e79120
2023-12-23 01:31:05.354 [WRN] CNCT: Unable to broadcast force close tx(f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296): transaction rejected by the mempool because of low fees: mempool fee not met: -26: mempool min fee not met, 3818 < 5120
yyforyongyu commented 10 months ago

Thanks for the logs, very helpful.

First of all, you cannot use 100 sats/vb as the fee rate because your local balance cannot cover it, as shown in this line,

2023-12-23 00:20:42.854 [ERR] PEER: Peer(REMOTE_PEER): unable to process close msg: unable to sign new co op close offer: initiator cannot afford proposed coop close fee

Given the size 278.5 vb and a fee rate of 100 sats/vb, you'd need at least 27850 sats as your local balance.

Secondly, this channel is in a conflicting state,

2023-12-23 00:20:42.836 [WRN] PEER: Peer(REMOTE_PEER): ChannelPoint(3c48e584920ac39d2868f0c466e20fc811afae81cd5911b709ec9183064bceb6:0) has status ChanStatusBorked|ChanStatusCommitBroadcasted|ChanStatusCoopBroadcasted|ChanStatusLocalCloseInitiator, won't start.

The status ChanStatusCommitBroadcasted means it's a force close, yet we have ChanStatusCoopBroadcasted which means it's coop, something we definitely need to investigate further. Also note that f7b6fe0c4958e5ad303dab547143d3a7354d9945950cbe217773f9f487448296 is a force close tx, not a coop, which means you cannot specify its feerate, which is why local_commit_fee_sat is used here.

Could you provide more details on how you used the command lncli closechannel? Did you use it multiple times with different args?

kingpalethe commented 10 months ago

Thanks for the info. Yes, I think I tried a cooperative close, got some kind of error, and then tried a force close.

ziggie1984 commented 10 months ago

I think we could improve the current coop-close negotiation, maybe it also affects the new design which will be introduced.

  1. We should check early on whether the desired feerate can be afforded, so that if the user finds out that the desired feerate cannot be met, he wants to keep the channel open? Currently we initiate the whole Shutdown process although we could already know whether we can afford the desired feerate or not.
  2. Same applies when the negotiation fails, why not just let the channel in status open and keep on using it, otherwise the initiating party will force-close it because he cannot coop close it if the FeeRange of his peer is not according to his desire and he can also not reactivate it because the channel is marked as ChanStatusCoopBroadcasted.

Wdyt ?

kingpalethe commented 10 months ago

Just wanted to add some more color to this from the perspective of a clueless user -- in case that is helpful.

I am continuing to find that closing channels with LND is a scary/unpredictable process due to fee problems.

Anecdotally, this is what I am finding:

-- If I use lncli closechannel without specifying a fee, I have seen that the broadcasted fee approaches 4x what is current on mempool.space ... for example: https://mempool.space/tx/0b06a77739f2e2d254b1ee9adb208f1bc723ac8c49bfe137ff2b38072599e426 -- for this close, I used lncli closechannel without specifying a fee. Presumably the negotiation with this peer yielded this 4x overpaid close fee?

-- If I use lncli closechannel and specify a fee in the ballpark of what mempool.space recommends, then it seems most of the time, the fee that is broadcast to the blockchain is significantly BELOW a good rate, and then the transaction get stuck. This happened to me a couple hours ago -- here is the transaction: https://mempool.space/tx/5c50d754dab82ebc0764c5baea1e5f02ce8152819fdf95d724567b9029765985 ... in this case I specified something like -sats_per_vbyte 60 .....

Note - I do understand that these fees are likely being pushed around in the negotiation process with the other node, so maybe it's not LNDs fault? Or at least not the fault of MY instance of LND?

Next: To remediate these "too-low-fee" closes, I have referenced these directions: https://docs.lightning.engineering/lightning-network-tools/lnd/unconfirmed-bitcoin-transactions#docs-internal-guid-5647dd03-7fff-dc71-47cf-5f7e2155a44d

I have tried commands like:

lncli wallet bumpclosefee --conf_target 6 b443b3da9b864ac140aa7567e3284b48c4fca304f77bf8e9a4a94bcc219f63c5:0 and lncli wallet bumpclosefee --sat_per_vbyte 60 b443b3da9b864ac140aa7567e3284b48c4fca304f77bf8e9a4a94bcc219f63c5:0

For both of these commands, lncli immediately returns without printing any response. And (as far as I can tell), nothing related shows in the LND logs.

If you would like me to grep the LND logs to try to see what LND did with these commands, please advise what I should search for.

thanks!

zynos commented 10 months ago

Looks like my problem when using ZeusLN. Look at the bottom for my log. https://github.com/ZeusLN/zeus/issues/1937

ardevd commented 10 months ago

Just chiming in with a similar issue:

Tried force closing a channel that has been offline for a long time and got the following error:

Unable to broadcast force close tx(8d8c20b37300eb389b3bc87360dff661c111089099add8db2c6a842efd0396cf): transaction rejected by the mempool because of low fees: mempool fee not met: -26: mempool min fee not met, 3100 < 6207

Trying to bump the close fee with lncli doesnt seem to work either:

$ lncli wallet bumpclosefee --conf_target 6 1f984c1dd285c684adbd592a008d99e4c9ce455d9537e797fdc5401ae10d6378:0
[lncli] channel not found

Any ideas?

ziggie1984 commented 10 months ago

2 options:

  1. Increase your local mempool size, the force-close tx does not make it into your mempool hence cannot be bumped either.
  2. Get the raw tx from your locs and broadcast it to mempools manually, then use chantools to bump the fee of this force-close.
ardevd commented 10 months ago

2 options:

  1. Increase your local mempool size, the force-close tx does not make it into your mempool hence cannot be bumped either.

  2. Get the raw tx from your locs and broadcast it to mempools manually, then use chantools to bump the fee of this force-close.

Thanks for the insight. I'll give it a go and see if it works out!