lightning / bolts

BOLT: Basis of Lightning Technology (Lightning Network Specifications)
2.11k stars 491 forks source link

Ability to set inbound and outbound fees #835

Open AbelLykens opened 3 years ago

AbelLykens commented 3 years ago

Taken from https://github.com/lightningnetwork/lnd/issues/4225 . Original by @SwissNode:

As a major channel balancer and node operator, it has become clear to me and the community of channel balancers I work with that we really need the ability to incentivize traffic in certain directions. As such, we would highly recommend having inbound and outbound fees to allow such incentivization.

Reply @Roasbeef :

As is, on the protocol level, you can only set the fees for outbound (routing an HTLC over that channel). The remote party controls the "inbound fees", since their bandwidth is used to carry the HTLC.

One thing we've discussed is possibly introducing "pair wise" fee schedules. So this would mean you can set different fees for A -> B, and A -> C, where B and C are your channels. We could possibly add some new TLV fields to the ChannelUpdate to make this possible, but no one has attempted to fully specify it yet.

and:

This is a spec level thing, so closing here as the discussion needs to be started on the rfc repo and/or the dev mailing list.

t-bast commented 3 years ago

I don't understand what that means. It doesn't make sense for you to set the inbound fees, that choice belongs to your peer.

dlaptev commented 3 years ago

that choice belongs to your peer.

At the moment yes, but it does not have to be that way. The fee in either direction can be a sum of what you take and what your peer takes. Being able to set both inbound and outbound fee will allow nodes to have a much better control of the liquidity flow.

t-bast commented 3 years ago

Ok that's what you meant, to have both peers collect a fee whenever a payment is relayed, regardless of the direction? I'm not convinced the incentives work though, can you make your case there? If I open a channel to you, I want to be able to set the relay fees and I want to collect the whole fee because I brought all the liquidity, I don't want to give you a cut of that fee. The only reason I would do that would be if I'm sure that the fee I'll also collect from your end of the channel exceeds the fee I let you collect from my side of the channel. But then we have an asymmetry and a loser here, so the equilibrium should revert back to one-sided fees.

dlaptev commented 3 years ago

I think yes, that's is how I understand the proposal.

I completely agree that the person bringing the liquidity should be rewarded more in the beginning. Not sure whether this should be handled explicitly though, I tend to think that the market will rule misbehaving peers out, but I could be wrong of course.

My case here would be longer-term. I have lots of channels where cumulatively the amount of forwards is >10x larger than the channel capacity. Note that the notion of "who brought the liquidity" kind of becomes blurry at this point.

Most of these channels have a dominant directions of forwards: some channels are mostly used as incoming channels, so most of time the balance is local, and some as outgoing channels, so most of the time the balance is remote. For the latter channels I increase the fees to "slow down" the flow of funds from me to the peer, and it works great: the channels become more balanced, the network benefits from liquidity both ways, and I collect some fees.

But I have no control over channels with predominantly incoming forwards. I usually drop the fees to zero to incentivize the flow "from me", but the liquidity is still mostly stuck on my end. Being able to set the "incoming fee" would serve exactly the same purpose as the outgoing fee for other channels: slow down the flow of funds in one direction and make the channels more balanced.

t-bast commented 3 years ago

But I have no control over channels with predominantly incoming forwards.

But in that case your peer has absolutely no incentive to let you lower his fees. It is (and IMO should be) entirely his decision to slow the flow of funds by lowering his fee or not.

I think that this proposal needs to be:

  1. More detailed, otherwise we can't accurately say whether it will even work or not
  2. The claim that it helps liquidity need to be much more substantiated (a hand-wavy "we feel it's better for liquidity" is not enough to motivate such an important change)
dlaptev commented 3 years ago

Agreed. I will let @Swissnode elaborate further if they had more detailed proposal in mind and quantitative evidence to support it.

Swissnode commented 3 years ago

The point of asking this is that currently I don't have a way of enticing movement in a certain direction along a channel. Certain channels basically drain sats so it would be nice to be able to set incoming fees of such channels to near zero to entice flow back down that channel.

This is still advantageous for me as a node, as I still obtain the outgoing fees for any routing through that lopsided channel

Hope that makes sense, it's actually a simple proposal...

AbelLykens commented 3 years ago

I see the same things @Swissnode sees. I want to balance my channels. I use https://github.com/accumulator/charge-lnd to set fees based on channel ratio (higher fees if most is on remote side).

If I am able to encourage routing IN by setting lower fees, that makes more sense to me...

t-bast commented 3 years ago

Sorry but no, that makes absolutely no sense to me (unless I'm missing something).

Are you saying that you are in a situation like A <---> B where you are A and most of the funds are on B's side? And you'd like to lower the relay fees B -> A to incentivize funds to flow back to A?

If that's the case it's a NACK from me. If all the funds are on B's side, it's their money, not yours (you have received the same amount of money in another channel before sending it to them on this channel). Since it's their money, it's 100% their decision, you should have absolutely no business deciding what relay fees they should set for their money. They will update their relay fees when they see fit; if you're complaining because you feel they're not managing their relay fees correctly, then simply don't open other channels to them (you don't care about this one, since all the money is on their side, you don't have anything at stake). If it makes sense in their strategy to get the funds to flow back to you, they'll lower their relay fees, but that should be entirely their decision.

Swissnode commented 3 years ago

image This explains it.

If I have 1 million sats coming in and I have an incoming fee and an outgoing fee on channels A and B as shown, I deduct the sum of those ppm's. In this case 25+75 is 100 sats for every million going through the path.

It doesn't matter what the channel balance is at any stage. I simply retain a certain fraction of the sats flowing along both channels.

In this way I can incentivise the usage of channel A by simply dropping the inbound fees to zero. There will still be an outgoing fee applied, but again I can incentivise the outbound usage of a certain channel by dropping ITS fees to zero.

This also allows any node operator to change his modus operandi. He could decide to NEVER charge outgoing channels and just charge certain incoming ones... Maybe I am connected to a node which emits way more sats than it receives, in which case you want to charge for using the incoming from that node...

t-bast commented 3 years ago

If I understand correctly your argument, I'm sorry but it's still a big NACK from me. To achieve your goal you're trying to control the relay fees that apply to funds that are fundamentally not yours. From your node's point of view, of course it would be better, but you're assuming what's best for your counterparties. This shouldn't be your decision, it should be a decision they reach independently, and the existing fee mechanism is well-suited for that.

begetan commented 3 years ago

It is possible to move the fee decision entirely to the right side. For example fee rate would be virtually splitted to 25 + 75 on the both channel B side.

So when a payment come from node A, the total fee would be 100 ppm. But if another payment come from node C (which is not present here) the total fee coud be just 75 ppm because "incoming" fee it is not set for this channel.

The question is it possible to propogate such data in the network?

openoms commented 3 years ago

@begetan this is what @roasbeef has mentioned as the pair wise fee schedules. In that case still only one peer would control the fee on a direction, but could set different (outgoing) fees depending on which peer the payment is coming from. This would also add a lot of extra parameters and communication burden, but would be a more viable way to achieve what is aimed in this issue which is to be able to prefer or penalize whole routes.

t-bast commented 3 years ago

In that case still only one peer would control the fee on a direction, but could set different (outgoing) fees depending on which peer the payment is coming from.

Ok I get it now, that makes more sense. You would provide discounted fees when it's coming from specific channels to nudge the choice of the overall route towards what helps balance your channels. This is worth exploring, but it's indeed probably quite costly in terms of gossip bandwidth used.

Swissnode commented 3 years ago

If I understand pairwise fees correctly, as your number of nodes, n, increases, the pairs increase (n-1)^2 whereas keeping inbound / outbound fees only increases by 2n (with less maintenance as you can set a default inbound and default outbound and just change when needed).

I am not exactly sure how fee determination currently works, but i assume the node trying to do pathfinding queries the fee across a certain channel. If inbound fees exist, it merely has to query two channel fees instead. Seems reasonable in network bandwidth, but maybe I don't understand how pathfinding works...

Swissnode commented 3 years ago

It is possible to move the fee decision entirely to the right side. For example fee rate would be virtually splitted to 25 + 75 on the both channel B side.

So when a payment come from node A, the total fee would be 100 ppm. But if another payment come from node C (which is not present here) the total fee coud be just 75 ppm because "incoming" fee it is not set for this channel.

The question is it possible to propogate such data in the network?

Yes you are correct that if it is simpler to think of, then only outgoing fees are affected, but the incoming node adds a fee onto this. I think t-bast was getting confused thinking that all inbound sats (which might not be owned by the node) are taxed. It is STILL an outgoing fee, only it takes into consideration the inbound direction to calculate, exactly.

C-Otto commented 3 years ago

Maybe it helps to think of this as a rebate?

"If you send a payment from A to my node and ask me to route to node B, of course you'd have to pay A's fees. But instead of charging you the full fees for the channel to node B, I'd be fine if you pay x% of that because you're helping me to balance the channel to A".

Raulo commented 3 years ago

This is an excellent idea. There are some nodes that generate a lot of incoming traffic and much less outgoing. There is currently no way to slow this inflow if the counterparty keeps the fees low. Either the channel balance gets moved to one side or there has to be a repeated rebalancing.

The @Swissnode proposal would allow auto-rebalancing, currently impossible with the fee management. With setting a proper fee, the inflow can be matched to outflow, ending the permanent imbalance. The @Roasbeef solution of pair fees would be equivalent but with more complication in the node management. Changing a channel fee would require changing all the pairs but it will work, too.

pgerstbach commented 3 years ago

I do like the idea of inbound fees very much. It only doubles gossip (if I am correct) but would really increase the network's ability to find an optimal equilibrium.

Currently I have peers that don't do a proper fee management, and I am "paying" for this with my low-fee-peers. I either have to increase their fees to protect them, or I have to close the channel that causes the problem. There are so many nodes that are not balanced at all, and this results in so many failed routing attempts. I suppose, this proposal would allow us to improve the success rate of payments throughout the whole network!

pgerstbach commented 3 years ago

I suppose this proposal (inbound fee, not pairwise) would have linear impact on routing, right @renepickhardt?

madeken commented 3 years ago

This is actually quite a clean and natural way to allow nodes to actually express the impact of routing funds through them.

AlbertoBoldrini commented 3 years ago

Is there any news on this? I think that this feature could be great for improving the efficiency of the network. If each node could specify two fees (ingoing and outgoing) for each of its channels and these fees could also be negative the problem should be solved and at the same time several interesting scenarios would appear. For example MyNode of the diagram setting a negative inbound fee from A and a lower outgoing fee to B could pay to balance their channel. In some scenarios balancing with negative fees could be more cheap than active balancing. End users could also benefit from negative fees while helping to maintain the network efficiency.

AlbertoBoldrini commented 3 years ago

Also positive inbound fees might make sense in some cases. Suppose that you are a big router node, and many unreliable nodes want to open a channel with you and forwarding some payments. You might be inclined to accept to keep your funds locked on their channels only for a given price that is your inbound fee.

Coinomatron commented 2 years ago

Perhaps it's an idea to make this optional in the protocol. Also worth considering the fewer gossip on failed payments could cancel out the extra gossip on channel fee negotiation.

xraid commented 2 years ago

whats perhaps is needed is a side protocol where peers can negotiate certain fees for certain times(balances) that would not necessarily be in the bolt spec but can be deployed at Node´s as applications signaling and set fees dynamically per a predefined agreement ... so a application layer can facilitate the functionality without changes to the spec today ...

tioneb22 commented 2 years ago

It makes sense to have the ability to control both the outgoing and incoming fees. @AlbertoBoldrini takes it a step further and suggests the ability to configure negative fees: your channel peer wouldn't pass on the opportunity to generate fees on both their own outgoing fee and the remote sides incoming fee! Controlling both fee's can greatly help balance the network. Has anyone built a proof-of-concept for this? How can I check the progress of this idea?

joostjager commented 2 years ago

I think to see the point of inbound fees, it is necessary to understand that not all incoming traffic is equal.

For example:

A node X has peers A, B and C. Peer A and B send a total of 1 BTC/day back and forth through X. X keeps earning routing fees without any direct cost. Because there is no direct cost, the routing fee that X charges can be low - 0.05%. X earns 50k sats/day in routing fees.

The next best route between A and B via another routing node costs 0.1%.

Peer C is an endpoint and only sends money to A and B via X, but never receives anything. The total amounts to 0.1 BTC/day. This means that X will need to keep opening fresh channels to A and B to earn routing fees. The routing fees will at least need to cover those costs. X doesn't have a lot of capital available and can only open channels of size 0.04 BTC. The average chain cost for a channel open+close is 5000 sats. This means that the routing fee will need to be at least 0.125%. X also wants to make a profit, so adds 0.05% on top of that making a total of 0.175%. X earns a net profit of 5k sats/day.

The problem now is that because there are only outbound fees, X will be forced to set the routing fee to A and B to 0.175%. Because that rate isn't competitive anymore for senders A and B, X will lose its prior business worth 50k sats/day. Total loss with C as the new customer is 45k sats/day.

The only thing that X can do currently is close the channel to C, so that X can maintain the low fee for traffic between A and B. If all routing nodes operate in this fashion, it means that C can only transact through a subset of nodes that charge higher fees across the board.

With inbound fees, the solution would simply be to keep charging 0.05% outbound to A and B. For peer C though, an extra 0.125% inbound fee would be charged to compensate for the unidirectionality of the traffic. Total profit 55k/day, plus node A and B paying only 0.05% instead of 0.1% for the next best route.

t-bast commented 2 years ago

I agree with the problem, but not with its solution, as inbound fees create other incentives issues and additional complexity.

The way I like to see it instead is that the inbound liquidity you get from someone opening a channel to you will eventually become (potentially idle) inbound liquidity you provide for them, and should be treated as such. If some node A opens a channel to you and only sends funds out, but never receives, once they've sent everything out, you are actually providing inbound liquidity to them that they should pay for.

The good news is that we already have a solution to make people pay for inbound liquidity: liquidity ads (#878)! I think re-using this mechanism to price these scenarios could make more sense and be simpler to manage than inbound/outbound fees. I haven't fleshed out the low-level details yet, but this is worth exploring IMHO.

joostjager commented 2 years ago

One way to look at liquidity ads is as an inbound fee that is paid only once and upfront.

This requires more trust on the side of the taker and doesn't allow dynamic adjustment if the traffic pattern changes. I'd expect that to create more incentive issues than adding a slight variation on the current fee structure by means of an inbound discount for certain channels.

I've explored implementation details in https://github.com/lightningnetwork/lnd/pull/6703 and can't say it is overly complex.

joostjager commented 2 years ago

Given the variety of opinions on this proposal, I opted to create a BLIP to underline the optionality of it.

https://github.com/lightning/blips/pull/18

niftynei commented 2 years ago

One way to look at liquidity ads is as an inbound fee that is paid only once and upfront.

I believe this is a false equivalence. A liquidity ad is a mechanism for advertising allocation of new capital / capacity along a novel edge. This action incurs real-world (onchain) costs (2x, open+close), and is an sorting operation on currently unallocated capital (will change post splicing? tbd).

Having people pay to route through you is a re-allocation of existing capacity along already established edges. There is no net cost to this (minus whatever people are charging to use/push their liquidity towards you).

t-bast commented 2 years ago

Also, the once and upfront part is incorrect, the lease renew mechanism isn't included in the initial version of liquidity ads, but it will very likely be added in the future.

Swissnode commented 2 years ago

I LOVE love love this proposal and have been requesting it for about two years (together with Alex B). Lightning missed out on two things. Dual-funded channels as the "default" and inbound fees. Let's hope we can rectify asap

joostjager commented 2 years ago

@Swissnode you're welcome to help testing the implementation in https://github.com/lightningnetwork/lnd/pull/6934

ProofOfKeags commented 1 year ago

Without opining too heavily on the solution: the problem is really that every time a node operator is routing a payment they are making a liquidity trade. They are trading liquidity on the outbound channel for liquidity on the inbound channel. Critically, liquidity is non-fungible from the operators perspective.

The liquidity they are acquiring on the inbound channel may be liquidity they can "move" (as in product), or not. It makes sense that if someone is "buying" your downstream liquidity that they either pay for it with a high fee and low quality outbound liquidity, or they pay for it with high quality outbound liquidity and a lower fee. Whether it makes it into the protocol or not, node operators MUST consider this trade. If there is no way to give this trade a more tunable and "true-to-cost" price (like you would be able with pairwise fee schedules), then it will manifest with a combination of higher-than-efficient fees and refusals to route.

Imagine a scenario where node operator has liquidity on Channel B which s/he has success moving with good revenue. The operator has two other channel partners A and A', A is a low traffic route on the outbound side, but A' isn't. In other words A' is "prime" liquidity. Here's the initial setup

A ||||||-------\
                O ||||||------- B
A'||||||-------/

In this case, when O considers the true cost of routing from A->B, they have to consider how long it will take to move the A-O liquidity back to the A side. O knows that the flow along A-B is fairly unidirectional. Now let's say they allow this to happen.

A -------||||||\
                O -------|||||| B
A'||||||-------/

Obviously O can no longer route A'->B because the outbound liquidity on B is exhausted. However, what's also true in this case is O can no longer route B->A' either. If A'-B had good bidirectional flow this is extremely costly and surely a scenario O wants to avoid. O has a few options to deal with this:

In all cases O's solutions are cost money or time. But of course, O is not going to lose money overall, so they compensate by raising their fees so that statistically all of those solutions are embedded in the cost they advertise to the network for routing on B. The other solution here is that O just says "no" to any onion from A that contains B as a downstream channel partner. This degrades the quality of the network either in terms of an overall higher fee level for everyone, OR with larger routing failure rates.

The following things have been suggested to alleviate this and they each have their own technical and social issues:

Any of these options improves the problem described above. The drawbacks of these solutions is it creates an overt protocol mechanism for discriminatory pricing that can be used for any reason the operator sees fit. Generally, the design of LN properly acknowledges that things have real world costs and that it is better to bring those decisions out into the open rather than them just happening in an opaque space...because they will.

Personally, I believe pairwise fee schedules are the most accurate way to price, but you do create the problems that arise with bartering and having to have N^2 pricing tables for things. Inbound fees sidestep this issue at the cost of slightly worse accuracy. Negative fees don't do anything to address the core problem, but does give us other means of coping with it.

Some or all of this may be obvious to the participants in this thread, if so, feel free to ignore, but as I was reading through it I wanted to type up a comprehensive review of how I understand the problem for posterity.