lightningnetwork / lnd

Lightning Network Daemon ⚡️
MIT License
7.67k stars 2.07k forks source link

Insufficient Bandwidth to route HTLC #6303

Closed Deekee62 closed 2 years ago

Deekee62 commented 2 years ago

Background

Failed forwards due to "insufficient bandwidth to route htlc" but there is enough outbound liquidity. Some knowledgeable noderunners have already cross checked and confirmed that this indeed is a strange behavior indeed.

Your environment

Hardware Details Hardware: BCM2835 Revision: d03114 Serial: 100000000b2d1319 Model: Raspberry Pi 4 Model B Rev 1.4

Steps to reproduce

Hard to say how this can be reproduced, at least one peer has reported that about 10% of his forwards show the same behavior.

Below some more details I’ve checked:

Below please find a 2 screenshot which document the behaviour:

telegram-cloud-photo-size-4-5819205817772128712-y

telegram-cloud-photo-size-4-5819205817772128713-y

Expected behaviour

Forwards should not fail if the is enough liquidity, see example above.

4'651K incoming remote 3'982K outgoing local 2'547K forwarding amount

Actual behaviour

Some forwards fail although sufficient liquidity is available.

Any assistance or guidance would be very much appreciated!

Many thanks in advance and have a nice day!

Roasbeef commented 2 years ago

There's a log line there about "violated target outgoing link", so that HTLC violated the policy for that link, but we send back the generic "insufficient bandwidth" error internally.

positiveblue commented 2 years ago

@Deekee62 also, just on top of the line @Roasbeef is talking about, could you paste the full line? I think the format is

insufficient bandwidth to route htlc: {amount to forward} is larger than {channel outgoing bandwidth} Are those numbers approx 2.5M and 3.9M?

Deekee62 commented 2 years ago

Thanks for your reply, unfortunately I don't keep logs that far back, but I will try to catch another example and report back. Do you need some more info or only additionally the above mentioned?

Deekee62 commented 2 years ago

Here another complete example:

HTLC_24638 image

1'176'859K outgoing local 899'820K forwarding amount

I hope the data provided is sufficient and have a nice day :-)

Roasbeef commented 2 years ago

In that screen shot, you can see that after accounting for the channel reserve, the channel doesn't have enough bandwidth to forward the HTLC. Look for "is larger than" in the screenshot.

Deekee62 commented 2 years ago

I'm sorry to bother you again and waist your precious time, but the channel reserve is only 40'000 sats, so there is still ~237039 sats of liquidity left!

image

I also have a problem to interpret the second amount mentioned in he line:

image

The difference shown is 813211530 mSAT

Many thanks in advance for an explanation and have a nice day!

Roasbeef commented 2 years ago

I'd check the other parameters of the channel, including the "max value in flight" field. What you've shown is what lnd used to make the forwarding decision. I recommend monitoring the logs more so you can see how things are accounted for. Also in the future screen shots of logs are harder for us to parse at times than using a paste bin.

Ripley81 commented 1 year ago

Hey Deekee62 - what command do you use to find that view with the HTLC? I suspect I have similar issues, or at least I'm not understanding the calculations involved when assessing routing ability.

chargelnd@raspibolt: ~ — ssh admin@raspibolt local — 251×22 2023-03-14 07-25-08