search

ElementsProject / lightning

Core Lightning — Lightning Network implementation focusing on spec compliance and performance
Other
2.85k stars 902 forks source link

fundchannel stuck with CHANNELD_AWAITING_LOCKIN:Funding needs 3 more confirmations for lockin. for a few days #2724

Open thestick613 opened 5 years ago

thestick613 commented 5 years ago

I'm unable to find the transaction id in any block explorer. Should i just close PEER force?

thestick613 commented 5 years ago

Normal close didn't work, so i closed with force.

I then got this in the listpeers output:

               "status" : [
                  "CHANNELD_AWAITING_LOCKIN:Reconnected, and reestablished.",
                  "CLOSINGD_SIGEXCHANGE:Negotiating closing fee between 183sat and 25747sat satoshi (ideal 4622sat)",
                  "CLOSINGD_SIGEXCHANGE:Waiting for another closing fee offer: ours was 7398 satoshi, theirs was 10175 satoshi,"
               ],

It dissapeared after a few hours, but the money still isn't there in listfunds, even after dev-rescan-outputs.

ZmnSCPxj commented 5 years ago

If the original transaction was never broadcast successfully, then the funding never actually happened... maybe?

How much funds did you put and how much is now in listfunds total?

thestick613 commented 5 years ago

Nope, false alarm.. found the money. So i tried to fundchannel with a peer, it failed (don't know the error message), then i funded some other channels. The first failed fundchannel still appeared in listpeers, so it got me confused.

I found an output of listpeers which shows it:

  {
     "id" : "0279c22ed7a068d10dc1a38ae66d2d6461e269226c60258c021b1ddcdfe4b00bc4",
     "connected" : true,
     "netaddr" : [
        "88.208.115.66:59542"
     ],
     "globalfeatures" : "",
     "localfeatures" : "82",
     "channels" : [
        {
           "state" : "CHANNELD_AWAITING_LOCKIN",
           "scratch_txid" : "8f82b2da0faae0e0d0ff0a22708d30e55308333e7886758399dbf5c3e4c5b8d5",
           "owner" : "lightning_channeld",
           "channel_id" : "d74c4fd3ddab103a862972ab66af4e0afa333aebada96ef4f9a6b5d5a3a9fa36",
           "funding_txid" : "37faa9a3d5b5a6f9f46ea9adeb3a33fa0a4eaf66ab7229863a10abddd34f4cd7",
           "private" : false,
           "funding_allocation_msat" : {
              "03c492f46d8e0a6256bb9c5c42f2aed24717f70a3f39e0961419dee511688e3110" : 0,
              "0279c22ed7a068d10dc1a38ae66d2d6461e269226c60258c021b1ddcdfe4b00bc4" : 5000001000
           },
           "funding_msat" : {
              "03c492f46d8e0a6256bb9c5c42f2aed24717f70a3f39e0961419dee511688e3110" : "0msat",
              "0279c22ed7a068d10dc1a38ae66d2d6461e269226c60258c021b1ddcdfe4b00bc4" : "5000001000msat"
           },
           "msatoshi_to_us" : 0,
           "to_us_msat" : "0msat",
           "msatoshi_to_us_min" : 0,
           "min_to_us_msat" : "0msat",
           "msatoshi_to_us_max" : 0,
           "max_to_us_msat" : "0msat",
           "msatoshi_total" : 5000001000,
           "total_msat" : "5000001000msat",
           "dust_limit_satoshis" : 546,
           "dust_limit_msat" : "546000msat",
           "max_htlc_value_in_flight_msat" : 18446744073709551615,
           "max_total_htlc_in_msat" : "18446744073709551615msat",
           "their_channel_reserve_satoshis" : 50001,
           "their_reserve_msat" : "50001000msat",
           "our_channel_reserve_satoshis" : 50000,
           "our_reserve_msat" : "50000000msat",
           "spendable_msatoshi" : 0,
           "spendable_msat" : "0msat",
           "htlc_minimum_msat" : 0,
           "minimum_htlc_in_msat" : "0msat",
           "their_to_self_delay" : 144,
           "our_to_self_delay" : 600,
           "max_accepted_htlcs" : 483,
           "status" : [
              "CHANNELD_AWAITING_LOCKIN:Reconnected, and reestablished.",
              "CHANNELD_AWAITING_LOCKIN:Funding needs 3 more confirmations for lockin."
           ],
           "in_payments_offered" : 0,
           "in_msatoshi_offered" : 0,
           "in_offered_msat" : "0msat",
           "in_payments_fulfilled" : 0,
           "in_msatoshi_fulfilled" : 0,
           "in_fulfilled_msat" : "0msat",
           "out_payments_offered" : 0,
           "out_msatoshi_offered" : 0,
           "out_offered_msat" : "0msat",
           "out_payments_fulfilled" : 0,
           "out_msatoshi_fulfilled" : 0,
           "out_fulfilled_msat" : "0msat",
           "htlcs" : []
        }
     ]
  },

Any idea what happened?

ZmnSCPxj commented 5 years ago

You did fundchannel, Something Wrong happened and the tx did nor confirm, you did dev-rescan-outputs (which marks UTXOs that have bern reserved for the previous fundchannel as unused), you did fundchannel to a different peer.

2696 might have fixed this.

cdecker commented 5 years ago

2696 should not fix issues concerning transactions getting stuck, rather it is meant as a way for us to debug and back wallets listing on-chain transactions.

If you have a channel funding that is stuck, you could do a dev-rescan-outputs, which gets the outputs used for the funding reset to available (if bitcoind kicked the funding out of its mempool). Then you can generate a new address addr using lightning-cli newaddr, and withdraw all available funds to that new address (doublespending the original funding and making sure it'll never confirm by accident), using lightning-cli withdraw [addr] all. Finally you can get rid of the entry in listchannels by using dev-forget-channel [chanid] and you're back to a clean wallet, without stuck channels.

3nprob commented 3 years ago

@cdecker Just to make sure - I have the same situation now (funding tx dropped out of mempool, fee way too low) - is the procedude you laid out above still accurate today?

cdecker commented 3 years ago

Yes, dev-rescan-outputs to reset the utxo state, then withdraw to a new channel (any input of the funding tx must be double-spent, all ensures that).

3nprob commented 3 years ago

Hm, that did not work as expected. For the record. I had already issued a (successful) withdraw prior to this but after the attempted channel open above. Plenty of confirmations for that one by now.

So it seems like my clightning got inconsistent now, believing it has two different txes with the same inputs.

Tried only dev-rescan-outputs so far:

The remote node also still sees it as pending.,

3nprob commented 3 years ago

Update: The operator of the counterparty of the failed channel is also interested in figuring this out. They're running LND and sent me this from their logs - ot seems like the node is still insisting on negotiating the channel:

2021-02-16 04:59:09.211 [DBG] LNWL: ChannelPoint(FAILED_TXID:0): Restoring 0 dangling remote updates
2021-02-16 04:59:09.211 [DBG] LNWL: ChannelPoint(FAILED_TXID:0): Restoring 0 local updates that the peer should sign
2021-02-16 04:59:09.211 [INF] PEER: NodeKey(OUR_ID) loading ChannelPoint(FAILED_TXID:0)
2021-02-16 04:59:09.212 [WRN] PEER: Unable to find our forwarding policy for channel FAILED_TXID:0, using default values
2021-02-16 04:59:09.212 [INF] PEER: Negotiated chan series queries with OUR_ID
2021-02-16 04:59:09.212 [INF] DISC: Creating new GossipSyncer for peer=OUR_ID
2021-02-16 04:59:09.212 [DBG] DISC: Starting GossipSyncer(OUR_ID)
--
 [DBG] CNCT: ChannelArbitrator(FAILED_TXID:0): new block (height=670810) examining active HTLC's
 [DBG] CNCT: ChannelArbitrator(FAILED_TXID:0): checking commit chain actions at height=670810, in_htlc_count=0, out_htlc_count=0
[DBG] CNCT: ChannelArbitrator(FAILED_TXID:0): new block (height=670810) examining active HTLC's
[DBG] CNCT: ChannelArbitrator(FAILED_TXID:0): checking commit chain actions at height=670810, in_htlc_count=0, out_htlc_count=0

---

[DBG] LNWL: ChannelPoint(EXPIRED_TXID:0): starting local commitment: (*lnwallet.commitment)(REDACTED)({
 height: (uint64) 0,
 isOurs: (bool) true,
 ourMessageIndex: (uint64) 0,
 theirMessageIndex: (uint64) 0,
 ourHtlcIndex: (uint64) 0,
 theirHtlcIndex: (uint64) 0,
 txn: (*wire.MsgTx)(REDACTED)({
  Version: (int32) 2,
  TxIn: ([]*wire.TxIn) (len=1 cap=1) {
   (*wire.TxIn)(REDACTED)({
    PreviousOutPoint: (wire.OutPoint) EXPIRED_TXID:0,
    SignatureScript: ([]uint8) {
    },
    Witness: (wire.TxWitness) <nil>,
    Sequence: (uint32) REDACTED
   })
  },
  TxOut: ([]*wire.TxOut) (len=1 cap=1) {
   (*wire.TxOut)(REDACTED)({
    Value: (int64) REDACTED,
    PkScript: ([]uint8) (len=22 cap=22) {
      REDACTED
    }
   })
  },
  LockTime: (uint32) REDACTED
 }),
 sig: ([]uint8) (len=70 cap=70) {
  REDACTED
 },
 ourBalance: (lnwire.MilliSatoshi) 0 mSAT,
 theirBalance: (lnwire.MilliSatoshi) REDACTED mSAT,
 fee: (btcutil.Amount) REDACTED BTC,
 feePerKw: (chainfee.SatPerKWeight) REDACTED sat/kw,
 dustLimit: (btcutil.Amount) REDACTED BTC,
 outgoingHTLCs: ([]lnwallet.PaymentDescriptor) <nil>,
 incomingHTLCs: ([]lnwallet.PaymentDescriptor) <nil>,
 outgoingHTLCIndex: (map[int32]*lnwallet.PaymentDescriptor) <nil>,
 incomingHTLCIndex: (map[int32]*lnwallet.PaymentDescriptor) <nil>
})

[DBG] LNWL: ChannelPoint(EXPIRED_TXID:0): starting remote commitment: (*lnwallet.commitment)(REDACTED)({
 height: (uint64) 0,
 isOurs: (bool) false,
 ourMessageIndex: (uint64) 0,
 theirMessageIndex: (uint64) 0,
 ourHtlcIndex: (uint64) 0,
 theirHtlcIndex: (uint64) 0,
 txn: (*wire.MsgTx)(REDACTED)({
  Version: (int32) 2,
  TxIn: ([]*wire.TxIn) (len=1 cap=1) {
   (*wire.TxIn)(REDACTED)({
    PreviousOutPoint: (wire.OutPoint) EXPIRED_TXID:0,
    SignatureScript: ([]uint8) {
    },
    Witness: (wire.TxWitness) <nil>,
    Sequence: (uint32) REDACTED
   })
  },
  TxOut: ([]*wire.TxOut) (len=1 cap=1) {
   (*wire.TxOut)(REDACTED)({
    Value: (int64) REDACTED,
    PkScript: ([]uint8) (len=34 cap=34) {
      REDACTED
    }
   })
  },
  LockTime: (uint32) 551182523
 }),
 sig: ([]uint8) {
 },
 ourBalance: (lnwire.MilliSatoshi) 0 mSAT,
 theirBalance: (lnwire.MilliSatoshi) REDACTED mSAT,
 fee: (btcutil.Amount) REDACTED BTC,
 feePerKw: (chainfee.SatPerKWeight) REDACTED sat/kw,
 dustLimit: (btcutil.Amount) REDACTED BTC,
 outgoingHTLCs: ([]lnwallet.PaymentDescriptor) <nil>,
 incomingHTLCs: ([]lnwallet.PaymentDescriptor) <nil>,
 outgoingHTLCIndex: (map[int32]*lnwallet.PaymentDescriptor) <nil>,
 incomingHTLCIndex: (map[int32]*lnwallet.PaymentDescriptor) <nil>
})

[DBG] LNWL: ChannelPoint(EXPIRED_TXID:0): Restoring 0 dangling remote updates
[DBG] LNWL: ChannelPoint(EXPIRED_TXID:0): Restoring 0 local updates that the peer should sign
[INF] PEER: NodeKey(REDACTED) loading ChannelPoint(EXPIRED_TXID:0)

They're suggesting to just call lncli abandonchannel on their end to abandon it, but we're not confident this won't result in further inconsistencies. I guess for good measure I ashould also recompile clightning with --enable-developer and issue a dev-forget-channel, but again, I'm not sure of that will just make it worse considering the cuplicate utxos from the same inputs.