lightningnetwork / lnd

Lightning Network Daemon ⚡️
MIT License
7.69k stars 2.08k forks source link

[bug]: LND-CLN err=update_fee XXXXXX unaffordable #8675

Closed Impa10r closed 6 months ago

Impa10r commented 6 months ago

Background

Running an LND node, have a non-anchor channel with a CLN peer. After the chain fees jumped, the two nodes can no longer maintain connection: state update from LND causes CLN to reply with warning "err=update_fee XXXXXX unaffordable" and hang up. A pending HTLC is at risk to cause an FC.

Log from my side:

2024-04-22 18:18:24.550 [INF] HSWC: ChannelLink(bf4d148a934a62fd02df33a0f57bb2d07bce16c45f37d4feb21f8acda6b529a3:3): Attempting to re-synchronize channel: SCID=827627:1245:3, status=ChanStatusDefault, initiator=true, pending=false, local commitment has height=32100, local_htlc_index=14878, local_log_index=16976, remote_htlc_index=330, remote_log_index=15207, remote commitment has height=32111, local_htlc_index=14878, local_log_index=16976, remote_htlc_index=330, remote_log_index=15207
2024-04-22 18:18:24.781 [INF] HSWC: ChannelLink(bf4d148a934a62fd02df33a0f57bb2d07bce16c45f37d4feb21f8acda6b529a3:3): received re-establishment message from remote side
2024-04-22 18:18:24.782 [INF] DISC: GossipSyncer(031633471d005f37252ede8114c6f4203e0a668270532f0f11a1e4072eb2eef272): applying new update horizon: start=2106-02-07 06:28:15 +0000 UTC, end=2242-03-16 12:56:30 +0000 UTC, backlog_size=0
2024-04-22 18:18:27.752 [INF] HSWC: ChannelLink(bf4d148a934a62fd02df33a0f57bb2d07bce16c45f37d4feb21f8acda6b529a3:3): sending 2 updates to synchronize the state
2024-04-22 18:18:28.006 [WRN] HSWC: ChannelLink(bf4d148a934a62fd02df33a0f57bb2d07bce16c45f37d4feb21f8acda6b529a3:3): received warning message from peer: chan_id=a329b5a6cd8a1fb2fed4375fc416ce7bd0b27bf5a033df02fd624a938a144dbc, err=update_fee 448000 unaffordable
2024-04-22 18:18:28.029 [INF] PEER: Peer(031633471d005f37252ede8114c6f4203e0a668270532f0f11a1e4072eb2eef272): unable to read message from peer: read next header: EOF
2024-04-22 18:18:28.029 [INF] PEER: Peer(031633471d005f37252ede8114c6f4203e0a668270532f0f11a1e4072eb2eef272): disconnecting

On CLN side:

"status": [
                  "CHANNELD_NORMAL:update_fee 448000 unaffordable",
                  "CHANNELD_NORMAL:Will attempt reconnect in 300 seconds"
               ],

The error persists since the halving and the update fee stays at 448000.

Your environment

Steps to reproduce

Chain fees go to 1000+ then back to 100+

Expected behaviour

The channel between nodes continue to work

Actual behaviour

The channel between nodes keeps disconnecting

I reported to both CLN and LND devs, but no one wants to take responsibility for this bug...

saubyk commented 6 months ago

This appears to be a duplicate of https://github.com/lightningnetwork/lnd/issues/7756 Has been fixed with https://github.com/lightningnetwork/lnd/pull/7805

Will be available with 0.18

Impa10r commented 6 months ago

This appears to be a duplicate of #7756 Has been fixed with #7805

Will be available with 0.18

Anything I can do today to avoid expensive FC?

saubyk commented 6 months ago

cc @ziggie1984 in case he has any ideas

ziggie1984 commented 6 months ago

The only way for this to recover is to wait until fees go down, or ask your CLN node to run a setting which discards the fee check.

CLN Setting:

--ignore-fee-limits=BOOL

JaviLib commented 6 months ago

The only way for this to recover is to wait until fees go down, or ask your CLN node to run a setting which discards the fee check.

CLN Setting:

--ignore-fee-limits=BOOL

I'm been told in the Telegram group that modifying this setting doesn't help. Is that right?

Impa10r commented 6 months ago

The only way for this to recover is to wait until fees go down, or ask your CLN node to run a setting which discards the fee check.

CLN Setting:

--ignore-fee-limits=BOOL

My CLN peer says he tried running with that setting. Fees at 100 are not that high. I use feeurl to feed more realistic estimates, not bitcoin core. Maybe CLN gets too high estimates (my bitcoin core thinks the fees are 1150 still), because they don't have such a setting.

JaviLib commented 6 months ago

CLN related bug: https://github.com/ElementsProject/lightning/issues/7254

ziggie1984 commented 6 months ago

From CLN Docs:

ignore-fee-limits=BOOL

Allow nodes which establish channels to us to set any fee they want. This may result in a channel which cannot be closed, should fees increase, but make channels far more reliable since we never close it due to unreasonable fees. Note that this can be set on a per-channel basis with lightning-setchannel(7).

see: https://github.com/ElementsProject/lightning/blob/master/doc/lightningd-config.5.md

Impa10r commented 6 months ago

From CLN Docs:

ignore-fee-limits=BOOL

Allow nodes which establish channels to us to set any fee they want. This may result in a channel which cannot be closed, should fees increase, but make channels far more reliable since we never close it due to unreasonable fees. Note that this can be set on a per-channel basis with lightning-setchannel(7).

see: https://github.com/ElementsProject/lightning/blob/master/doc/lightningd-config.5.md

Made no difference, same error ((

ziggie1984 commented 6 months ago

Can you report detailed logs of both nodes while this setting is active.

Roasbeef commented 6 months ago

Asked for some node details here: https://github.com/ElementsProject/lightning/issues/7254#issuecomment-2073832114

Impa10r commented 6 months ago

Can you report detailed logs of both nodes while this setting is active.

My CLN peer did not send me his logs. My LND logs did not show anything different. The HTLC just expired and the channel FCd (((

JaviLib commented 6 months ago

Asked for some node details here: ElementsProject/lightning#7254 (comment)

I answered you there: https://github.com/ElementsProject/lightning/issues/7254#issuecomment-2076920748

BTW I think we may have found another related bug in LND: when feeurl cannot be retrived, it seems it defaults to 253sat/kw, which is ridiculous. This might be the problem, plus unable to renegotiate with CLN once the new low fee is stablished.

I will open another issue.