forwarded payments and the diner's dilemma

[michielbdejong]

When deciding on what transaction fee to charge, a connector will take into consideration how much money it wants to earn from each transaction (an argument for charging higher fees), as well as how much business it expects to attract (an argument for charging lower fees).

Attracting business can work in two ways:

• in the short run, give a competitive quote and have your peer believe that you generally succeed at delivering payments at whatever price you quote
• in the long run, have your peer believe that not only do you generally succeed in delivering a payment, you also generally charge a low fee

In cases where the sender send a little bit more than strictly necessary, both are more complicated when using forwarded payments (without explicit destination amount in the packet) than when using exact payments (with explicit destination amount in the packet). This is due to the diner's dilemma.

The analogy of ordering more expensive food if the bill is shared in the diner's dilemma is that of charging a higher fee if not you as a connector, but all connectors as a group, are made responsible for how much arrives at the destination for a given source amount.

The analogy is not 100% because in the diner's dilemma, everybody can see what the others are ordering, and in forwarded payments, all you see is that one person ordered the lobster, not who it was.

[emschwartz]

Do you still think it would play out this way if:

1. The game is played many times
2. The value of each one is low
3. It's not a group of friends going out to dinner but instead each person is choosing to bring along one other friend, who chooses one of their friends, and so on (so it's not about blaming the person who ordered the lobster but just about blaming the friend that you brought)
4. There are multiple paths the payment could take (if we don't have options connectors don't need to do anything tricky to raise their prices)
5. We adjust who we bring to each successive meal based on the results of all the previous ones?

[michielbdejong]

Each time I forward a payment, I can choose who I forward it on to, and how much commission I charge myself. The information I get back is whether that payment succeeded or failed. If a payment succeeded, I think, I was too careful, I could have charged more. If it failed, I can seek one of six explanations:

1. someone before me charged too much
2. I charged too much
3. the person directly after me charged too much, but will correct this next time
4. the person directly after me charged too much, and will not correct this next time
5. someone after that next person charged too much
6. the receiver was never going to fulfill this payment anyway

In cases 1., 3., and 6., there is nothing I have to do. In case 5., someone after me should adjust their routing. In case 2. I should adjust my pricing. Only in case 4. I have to change where I forward payments to next time.

An easy way to make all of this easier is to mention the amount in the packet. Then, me and my onward peer never have to argue about how much they charge and how much I charge, it will all be visible and measurable.

Maybe one confusion I realised just now: We want to introduce amount-less payment because it relieves the connector of the task of knowing which outgoing amount it should pay depending on the destination amount on one of potentially millions of ledgers. But this is not the case; a simple connector can still just apply the same rate as the one they apply when passing back remote quote responses. The fact that the amount is in the packet doesn't stop them from using that strategy.

And for the sender, they can do probe payments with varying destination amounts, using binary search to quickly find the sweet spot with a good success rate. You could also send one amount-less payment as a first probe, and use that as a starting point, slowly increasing the destination amount per chunk in the packet until you have the right trade-off between throughput and exchange rate (see also #333 for a musing about how the whole thing ends up acting like an order book: if you want your payment to complete quickly, cross the spread; if not, don't)

[emschwartz]

I think it's easier to think about if you don't assume there is a minimum the receiver will accept. Connectors forward payments at whatever rate they choose and in most cases the receiver fulfills the payment (communicating back to the sender how much they got). If a given connector and its chosen peers are the cheapest path, they get the next couple of payments. If they aren't, they don't. You can apply whatever kind of dynamic pricing algorithm you want, but it really comes down to whether the sender or the previous connectors know of a cheaper route at that moment.

The reasons I think amount-less packets are important are:

1. The open Interledger will inevitably have some Maximum Payment Size
2. The higher you assume the Maximum Payment Size to be, the more limited the set of connectors will be
3. I want to build a super competitive network with extremely low rates
4. Building for lower Maximum Payment Sizes makes the network friendlier to smaller connectors, helps connectors reduce risk (and thus prices), and allows us to simplify the core protocols even more
5. Even if the MPS is low, people will still want to send larger payments, so we will need some method for chunking the payments (otherwise you will just get an unrecoverable error when you try to send an amount that's too large, like we often ran into on the ILP Kit lunch network)
6. If the MPS is very low (and thus the network would be open to very small connectors), most payments will end up being chunked payments (or small chunks that are part of a streaming payment for an ongoing service)
7. If most Interledger payments are tiny parts of a larger payment or interaction, there's no point in setting up the network to "deliver" specific amounts anyway, because you always want the connector to deliver as much as possible. (Getting rid of delivery also means connectors don't need to know when an address is "local", so it makes Interledger addresses much more flexible. In a forwarding-only system, it's perfectly fine for someone to be addressed in terms of a bigger connector and have their address start with the connector's address, even if they have an exchange rate that isn't 1:1.)
8. Connectors may try to forward as little as possible and the only thing that will keep the fees low is competition. (It doesn't matter if payments are rejected because they amount that was delivered is too low. If that happens and there are no alternate routes, the sender will just increase the amount.)
9. As I mentioned before, building for smaller payment sizes makes the network more competitive

[michielbdejong]

I think it's easier to think about if you don't assume there is a minimum the receiver will accept.

that's an interesting thought experiment. I think that way you can show very well that the diner's dilemma occurs, and that it's necessary to disclose, at least to the previous and/or next connector, who is charging how much (either that connector's fee, or the sum from that connector to the destination, or the sum from the source to that connector).

If a given connector and its chosen peers are the cheapest path, they get the next couple of payments. If they aren't, they don't.

that works if the paths all start at the sender and end at the receiver without overlapping. if there are forks halfway, there's not much the sender can do to influence the choice of path there.

1.-7. are a compelling argument for why it's ok if the destination amount is unpredictable, but not for leaving it unspecified.

8. seems to be an argument for my standpoint, that the diner's dilemma is going to be a problem in an open network.
9. is nice, but also doesn't give a reason to remove the amount from the ilp packet.

[emschwartz]

Is your main argument just that connectors have an incentive to raise their rates as high as they can? If so, I agree with that.

What I don't understand is why you think having the amount in the packet changes the dynamic. In ILP 1, you get a quote using ILQP first. If a connector takes more than they quoted the payment will fail. I think this is what you're focusing on. However, what stops a connector from just quoting a worse rate and delivering on that? If there's no competition, you don't need to be sneaky about charging a higher rate, you can just do it because there's nowhere else to go.

that works if the paths all start at the sender and end at the receiver without overlapping. if there are forks halfway, there's not much the sender can do to influence the choice of path there.

Everyone is subcontracting out the rest of the path to other people. If your subcontractor picks bad sub(sub)contractors, it'll make you look bad. Everyone should take whatever steps they need to to ensure that they pick the best subcontractors available.

1.-7. are a compelling argument for why it's ok if the destination amount is unpredictable, but not for leaving it unspecified.

What do you mean by "unpredictable" vs "unspecified"? Is unpredictable just when the connectors can't see the amount and unspecified when the receiver doesn't know the amount?

9. is nice, but also doesn't give a reason to remove the amount from the ilp packet.

What is the amount in the ILP packet for? Why would I want to use that if I always want you to pass on as much as little as you think you can get away with and I'll decide from experience whether you're the best connector or not?

Also, if I want to use a test payment (either one that will fail or the first chunk of a larger payment that will be fulfilled) to figure out a path's exchange rate, how do I set the destination amount? Having the amount in the packet introduces a requirement to use a separate quoting protocol first. You cannot just start sending payments.

[michielbdejong]

What is the amount in the ILP packet for?

I really liked your argument during our phone call, that a connector doesn't care about source or destination amount, they just care about their local rate (higher for more profit per payment, lower for more payments). And so you could indeed say let's just get rid of the amount in the packet, but I still think the diner's dilemma becomes a problem, then.

Suppose we want to smuggle boxes of cigars past 10 borders. The boxes are sold to the next smuggler at each border. Each smuggler is allowed to set their price, and through out-of-band communication the markets stabilize at one cigar per border crossed. So each smuggler gets to take one cigar out of each box they pass on. So out of a 100 cigar box, there should be 90 left at arrival.

Say the 7th smuggler takes out two cigars instead of just one, and instead of 90 cigars, the receiver sees only 89 cigars arrive. This is not stealing, this is just his new local rate. As soon as the receiver detects this, they alert the sender, and the sender picks another first smuggler. The first smuggler doesn't really know which of the boxes caused the drop in traffic, and so he cannot work out which second smuggler to penalize for it. Meanwhile, the new first smuggler may still be forwarding the boxes via a route that still includes the 7th smuggler who doubled their price!

So that's not going to work, we will need some other mechanism to react to one smuggler raising their price, that penalizes that one smuggler, not all ten smugglers that happened to have been involved in the box which arrived less full. That's the analogy with one diner ordering lobster for himself, because the bill for that is shared anyway; this lack of coordination leads to everybody ordering lobster, because everybody else is also doing it. The transaction fees on the Interledger would thus be higher, which would in the short term be good for the connectors, but bad for the users, and thus for adoption.

You could think of various logging and source routing mechanisms to fix this, but before discussing the fix, do you agree that we need a mechanism where the price set by one connector affects only the traffic through that one connector, and connectors don't "share the bill"?

So if we agree on that, I think we can fix this by putting the destination amount in the packet, even if it's only to make this accounting mechanism possible. A simple accounting mechanism is if each smuggler opens each box to count the cigars. That way, the 8th smuggler will immediately notice that it contains one less cigar than what he usually sees. He will understand that either he needs to swallow the loss (take zero cigars himself), or pass the underfilled box on to the 9th smuggler, both of which are unattractive options for him. Therefore, he will just reject the box with an InsufficientCigarsLeft error. ;)

The 7th smuggler can now immediately repent his decision to raise his fee, because he will have to embarrassingly pass back the error to the 6th smuggler, even if it was totally clear that the 6th smuggler did pass sufficient cigars to the 7th. So now the 6th smuggler knows that the 7th one is not to be trusted.

Interestingly, there would be several ways to define what an "underfilled" box of cigars is at border number 7: either you say, it needs to be at least (90+3), if you know the destination amount, or you say it needs to be at least (100-7), if you know the source amount.

If not all boxes start at border 1 and finish at border 10, you could even have an unrelated unit of value, where you say "this is a size 95 box", and at each border, smugglers know how much they can get for a size 95 box. It would mean "a box that is worth between 100 and 90, depending on at which border it's traded". Then, you could even compare boxes that travel a shorter or longer distance (e.g. start at border 2 and have their destination at border 7).

So you don't need to have the value at the destination ledger in the packet, but you do need some sort of packet value indicator, so that underfilled boxes can get detected immediately.

[emschwartz]

I think you're bringing up fair questions about how connectors should determine what peers to forward payments to. However, I think you are discounting a very important part about how ILP currently works and focusing too much on the payment phase rather than the quoting phase. I don't think having the amount in the packet solves the issues you're raising in the way you think it does.

---

Each smuggler is allowed to set their price, and through out-of-band communication the markets stabilize at one cigar per border crossed. So each smuggler gets to take one cigar out of each box they pass on. So out of a 100 cigar box, there should be 90 left at arrival.

This is not how ILP works.

First, the sender asks one or a couple of smugglers in their neighborhood how many cigars they think will get to the destination if they send a box of 100. The smuggler has either collected this information up-front or needs to go and ask the next smuggler in their route. Either way, any smuggler in the path can set any price they choose and they will always have an incentive to set their rates as high as they think they can get away with. There is no reason to assume that each smuggler will take 1 cigar, or that this rate will stay consistent for any length of time.

Your conclusion about the usefulness of the amount in the ILP packet seems to rest on the assumption that what you're trying to identify is which connector took more than their quoted amount. However, the more important thing to look at is how they set the prices they quote. If there's a diner's dilemma, I think it appears just as much in the quoting stage of how ILP currently works as it would if we used payments for quotes.

The first smuggler doesn't really know which of the boxes caused the drop in traffic, and so he cannot work out which second smuggler to penalize for it.

The first smuggler sent the box on to exactly one other smuggler, so they know exactly who to penalize for bad rates: the second smuggler. You can never penalize someone further down the path than your direct neighbor.

do you agree that we need a mechanism where the price set by one connector affects only the traffic through that one connector, and connectors don't "share the bill"

No. Every connector chooses who they pass the payment on to and take full responsibility for every successive connectors' bad behavior.

So you don't need to have the value at the destination ledger in the packet, but you do need some sort of packet value indicator, so that underfilled boxes can get detected immediately.

What you really want is a way to tell if a quote you get has a good or bad rate, but there's only one way to do this: compare it to someone else's quote. If you don't have someone else to ask, you are out of luck and you just need to decide whether you want to send the payment at that rate or not.

---

How connectors choose which peers to use is a good question and it'll be a complex and evolving answer. I think they'll use some mix of historical experience, heuristics, broadcast information, and test payments to figure this out.

However, whether there is an amount in the packet or not will not help them, because it can only be used to identify when someone takes more than a certain expected amount, not when someone is just leading you to expect a bad rate.

[michielbdejong]

This is not how ILP works.

ok, let's also consider the quoting step here. Also, I forgot to clarify that at each border, there are at least 2 smugglers who could do the job, otherwise indeed that single smuggler could set any price they want:

We want to smuggle boxes of cigars past 10 borders. The boxes are sold to a next smuggler at each border. Each smuggler is allowed to set their price, and compete with other smugglers who operate at the same border as they do. Either through route updates with price information, or through remote quoting, the markets stabilize at one cigar per border crossed. So each smuggler gets to take one cigar out of each box they pass on. So out of a 100 cigar box, there should be 90 left at arrival.

Scenario 1: one of the smugglers at the first border increases their rate in their route broadcast and/or quote response. Would they be able to earn more money by doing so? You say:

the more important thing to look at is how they set the prices they quote. If there's a diner's dilemma, I think it appears just as much in the quoting stage of how ILP currently works as it would if we used payments for quotes.

But I would argue there's no diner's dilemma there; the smuggler who increases their price will immediately lose all business, unless that price increase is tiny and they make up for the slightly higher price in their fame for being reliable and fast. Suppose at the first border there are 5 smugglers, who each respond 90 if you ask for a QuoteBySource with sourceAmount 100. Now one of them suddenly starts responding 85, in an attempt to increase their own profits. They know they will only have to pass on 94 cigars for 85 of them to arrive, and will be able to keep 6 instead of 1 cigars for themselves. So the sender gets four quote responses which say 90, and one which says 85. They now need to choose which first smuggler to send through. For every box they send through the more expensive smuggler, they expect only 85 instead of 90 will be left when they arrive at their destination, and so that's not an attractive offer. The sender will reduce the number of boxes sent through that more expensive smuggler, as an easy way to increase their overall cigar throughput. There is no "shared bill" effect there, if you give a worse quote response than your competitors at the same border, you will simply be excluded from the game. A simpler example is maybe where this smuggler would respond 105 instead of 100 to a QuoteByDestination with a destinationAmount of 90. It just will not result in any traffic going through them.

So suppose all smugglers charge a rate of 1 cigar per border crossing. Remember, there are 10 borders to cross, and several smugglers at each border competing with each other. Now read https://github.com/interledger/rfcs/issues/340#issuecomment-344902645 again.

[adrianhopebailie]

While this was a fun exercise I'm not it helps us to make any useful decisions.

Firstly, the diners dilemma is simply a bad analogy. If the diners were following ILP it would be something more like:

Alice wants to get a burger for Bob so she invites Chloe to dinner who invites Dennis who invites Bob. When the bill comes at the end it's discovered that the money Alice brought along to pay is not enough for Bob to get his burger so everyone goes to the bathroom and throws up their dinner and they leave without paying. Alice goes to find someone else to invite for dinner that won't be as greedy as Chloe and her friend Dennis but will still ultimately bring Bob.

Secondly, in the cigar smuggler scenario, you assume that the smuggling network operates in a data vacuum. Sure, a smuggler can try to take more than their fair share and it will not be immediately apparent who in a long chain was responsible but let's be realistic:

1. Payments are unlikely to follow chains of more than a few hops otherwise they're unlikely to be cost effective anyway.
2. In a digital system where data is being exchanged at high speed the ability to isolate a bad actor will be swift and the risk of being singled out and ejected won't be worth the cost.
3. @emschwartz makes a good point about choosing your sub-contractors so you can deliver a good service to your customers. This applies all the way down the chain. If your immediate next connector is delivering a poor service then you will threaten to stop sending her business and she'll figure out why her next connector is delivering her poor service and so on.

Everything you are saying is assuming a broken market where, for some reason, market forces have broken down and competition is impeded. I find it hard to believe that this will be the norm and I definitely don't think we need to design for that at the interop layer.

As a general comment I think trying to predict this kind of thing in a system that has no obvious equivalent on which we can model it is pointless. We are not economists and we have no data to base any of these assumptions on.

The best thing we can do is build the network and test these assumptions with real money. It's all speculation until there is skin in the game.

[michielbdejong]

The diner's dilemma only models the connectors, not the sender or the receiver.

You're right that data will be exchanged to isolate a bad actor. The "box size" concept I explain at the end is that data. And destinationAmount is one implementation of measuring box size. It's essential that we don't remove that (unless we replace it with sourceAmount or with amount relative to some well-known unit of value).

[adrianhopebailie]

And destinationAmount is one implementation of measuring box size. It's essential that we don't remove that (unless we replace it with sourceAmount or with amount relative to some well-known unit of value).

I disagree. I don't believe having that data makes any difference to the intermediaries.

If the sender wishes to express how much they intended the receiver to get they can pass that in the payment data that is encrypted and not seen by the intermediaries.

[michielbdejong]

I don't believe having that data makes any difference to the intermediaries.

Hm, right now I can't think of a clearer way to explain it to you than with this comparison to boxes of cigars.

By "the box is underfilled" I mean "the box contains fewer cigars than expected, given its current position / progress through the network, and its box size".

By "box size" I mean a slightly more abstract concept than destination amount, it's more a number that's proportional to a weighted combination of source amount and destination amount.

Boxes are either already underfilled when they're passed along from one smuggler to the next, or become underfilled later along the chain. Passing on an underfilled box is a bad deal for a smuggler, because it's likely to be rejected by the receiver. In order to determine if a box is underfilled, it's not enough to know how many cigars it contains, you also need the data (the data which you believe makes no difference) of how many cigars are supposed to be in the box.

[adrianhopebailie]

But in ILP an intermediary has no idea how far they are from the source or destination so they have no idea if the box is underfilled or overfilled.

The decision about whether or not to route a payment and how much to put in the outgoing transfer can only be based on historic data about payments on the same route or external data about a fair rate of exchange.

Your whole theory is based on assumptions around connectors taking more than their "fair share" of a payment if they can get away with it that I don't agree with in the first place.

[michielbdejong]

in ILP an intermediary has no idea how far they are from the source or destination so they have no idea if the box is underfilled or overfilled.

That may be true for what #312 proposes, but it's not true about Current Interledger, where each connector knows the exact liquidity curve between their incoming ledger, their outgoing ledger, and the destination ledger.

The decision about whether or not to route a payment and how much to put in the outgoing transfer can only be based on historic data about payments on the same route or external data about a fair rate of exchange.

Again, I think you're not talking about Current Interledger, but more about the alternative situation which #312 proposes.

assumptions around connectors taking more than their "fair share" of a payment if they can get away with it that I don't agree with in the first place

If you think connectors in a network will all act fairly then indeed you don't need any of these protocols; in fact, then you don't even need hashlocks, validators, or third-party ledgers.

[momerath42]

Imagining connectors finding ways to buy lobster (or just take a little more than every connector in a chain could without a payment failing) is one of the scenarios that led me to build the simulation I did. It's also why I argued that route-selection will have to rely on complex heuristics.

In the two-cigars instead of one example, I think we can expect relatively simple heuristics to route around the greedy connector, within some number of rounds. The minimum number of rounds, I would expect to go up with the path length and number of possible routes a payment might have taken that are indistinguishable to a given connector. If more than one connector is "playing games" - for instance, they infer that a neighbor of theirs took two cigars last round, and that if they take two this round, it will be difficult for the upstream peers to know which of you it was, I expect the time-to-confidence to become unwieldy.

As for including an amount metric in the packet: it's seems potentially somewhat useful, but my gut says there are many more games to be played that it won't help much with. I think we should extend my simulation to model the alternatives here, and get some adversarial agent action going (reinforcement learning would be good, but we could even just hand-craft a few strategies and heuristics and see what happens).

[sappenin]

...When the bill comes at the end it's discovered that the money Alice brought along to pay is not enough for Bob to get his burger so everyone goes to the bathroom and throws up their dinner and they leave without paying...

@adrianhopebailie I think this means I shouldn't try to pay with Interledger next time we're out to dinner together.

:)

[michielbdejong]

Just talked to Evan about it in our 1:1, his assumption is that each smuggler will regularly do test payments from themselves to the receiver, and base their choice of onward path on that. So if each smuggler is able to do such test payments, then it's correct that they don't have a need to also look at statistics from real payments (although of course, the biggest data set would be if a smuggler could have both test data and live data to base their routing decisions on).

[adrianhopebailie]

each connector knows the exact liquidity curve between their incoming ledger, their outgoing ledger, and the destination ledger

In theory, but in practice we have found this to be very difficult to achieve. Current Interledger has proven to not be scalable so I don't see the point in trying to design for a scaled up version of it.

I strongly support the proposal in #312 to drop the amount field as I don't think it should be used by intermediaries in their routing decisions. Further, if it is used, it creates a vulnerability for those intermediaries that rely on it as nothing stops other intermediaries from gaming that value.

I still think we are wasting our time theorizing about this stuff when the only way we will know how a real market is going to react is if we start to deploy a real network.

The core ILP protocol needs to be as simple as possible. Handling alternative network behaviours should be handled by transport layer protocols (i.e. there could be a transport layer protocol that reveals the destination amount and that could prove to be very popular but it's not needed in ILP).

[sharafian]

I agree with Michiel in that the sender's worst acceptable rate can be surpassed if we don't have an amount specified in the ILP packet. But I agree with Adrian and Michael that we should see how these incentives work empirically, especially because this is actually an end-to-end decision and doesn't influence the design of ILP.

The reason the amount was in the ILP packet was:

1) to allow the connector to see the final amount for delivery purposes (or for greedy connector behavior, I guess). 2) for the receiver to look at it and make sure its incoming transfer amount matched the packet amount.

2 is required if you want to guarantee a rate, but because it's a check done by the receiver we can still implement it with end-to-end data only. The connectors don't need to be able to look at the amount for it to work.

You could then ask how the sender determines the destination amount. That could be done with an unfulfillable payment, or the sender can look at each completed chunk of the payment and set the next packet amount based off of how much got there.

Based on that, I can't think of any compelling reason we should keep the amount as a required part of the packet.

[michielbdejong]

Response to @adrianhopebailie:

in practice we have found this to be very difficult to achieve. Current Interledger has proven to not be scalable so I don't see the point in trying to design for a scaled up version of it.

Wait, I'm not saying Current Interledger is ready for production yet, because @emschwartz does have a fair point that if we rely on live payments for stats, competing onward connectors will try to mess up each other's stats, and that's a problem in Current Interledger which needs fixing. But as someone who's been involved in the implementation of the current liquidity routing scheme, as well as in the efforts to scale the community network, I don't agree at all with either of those statements (assuming you're talking about the community network), nor with your conclusion.

we are wasting our time theorizing about this stuff

This issue was about something that didn't work in theory; if something doesn't even work in theory, then it's not very useful to try it out in practice. But the situation changed yesterday, as @emschwartz explained the missing ingredient in my model, namely every connector doing periodic test payments to every destination ledger. So now I agree with you, we can just stop theorizing and try it out. But just trying out the happy path would create an illusion of success; we should try it out including rational (i.e., selfish) agents in the network, and including those test payments.

there could be a transport layer protocol that reveals the destination amount and that could prove to be very popular but it's not needed in ILP

If we take test payments into account, then yes I agree with that statement.

[michielbdejong]

Responding to @sharafian:

this is actually an end-to-end decision

True, whether or not to fulfill the (final) destination transfer, given its transfer amount, is a decision in which only the sender and the receiver have a say, and the intermediate connectors don't need to be asked for their opinion.

But in Current Interledger, the packet amount has a second function: suppose connector 1 forwards a payment to connector 2. Connector 2 has previously made promises (in a route broadcast or in a quote response) about its exchange rate / transaction fee. Connector 1 can now, given the destination amount, adjust the transfer amount to hold connector 2 to that promise. That means the promise obtains meaning. And once promises in route broadcasts and quote responses have meaning, you can base routing decisions on them. And unless you add test payments (which Current Interledger doesn't use), then that's essential.

Conclusion: amount in the packet is only necessary if you don't do test payments. If you do do test payments, and assuming your test payments force onward connectors to treat you nicely in all payments, then the amount in the packet is not necessary, and the diner's dilemma does not apply.

[justmoon]

Apologies for resurrecting a closed ticket, but I don't totally agree with the conclusion here and you referenced this thread, so I figured it would be best to respond here.

---

First, I totally agree about the fact that connectors in ILPv4 have a need to figure out whether their peers do a good job. This requires what you call "test payments". I think in practice this would have to be a combination of manual testing, automated testing, reports from customers about real traffic, and so on.

What is not true is that a public destination amount removes this requirement. In other words, ILPv1 needs "test payments" just as much as ILPv4 does.

But first, let's address the idea of "promised" rates:

Connector 1 can now [...] hold connector 2 to that promise. That means the promise obtains meaning.

In your view, quotes and route liquidity curves are promises. If I publish a certain rate curve and then execute at a different rate I have broken my promise and may be punished (by depeering etc.)

When quotes are not merely informational, but guaranteed rates (for some period of time), then they are options, so the free option problem applies. Unlike prepared packets in ILPv4 though, they aren't necessarily short-lived and the amount isn't necessarily small, so the option value could be significant.

Let's say we get rid of the liquidity curves in route broadcasts, only cache quotes for a very short time and we add a fee to ILQP. Would this work?

It might, but the amount in the packet would still not have any benefit. To understand why that is we must first understand that success rates in ILP are meaningless:

Suppose Connector 1 forwards a packet with the correct amount according to Connector 2's quote. From there, there are exactly two possibilities:

• Connector 2 may return a fulfillment which could be because Connector 2 forwarded the packet correctly. Or it could mean that Connector 2 is in league with the sender and was able to fulfill it without ever forwarding the packet. The fulfillment data is controlled by Connector 2, so nothing in the fulfillment data can help us determine if Connector 2 is evil. In other words, a successful payment does not give Connector 1 any information about whether Connector 2 made good on their promise.

• Connector 2 may return a rejection (or nothing), which could be because Connector 2 took too much of a fee. Or it could mean that Connector 2's competitor is in league with the sender and the payment was never fulfillable. The rejection data is controlled by Connector 2, so nothing in the rejection data can help us determine if Connector 2 is evil. In other words, a rejected payment does not give Connector 1 any information about whether Connector 2 made good on their promise.

Statement 1: Therefore, Connector 1 can never infer anything about the honesty of Connector 2 based on its observations.

Now you may say that Connector 3 could see if the rate that Connector 2 is asking of them is correct. After all, if Connector 2 is charging too much, the packet would be too small for Connector 3 to forward. But Connector 2 can simply use the free option trick. If the rate moved in their favor, they forward the packet to Connector 3, if it hasn't, they drop it blaming Connector 3. Connector 3 never sees anything that they know is malicious behavior on the part of Connector 2. And neither does Connector 1 due to Statement 1. If Connector 1 talks to Connector 3, 4, 5, 6 and so on, they can figure out that some connector is lying, but not who.

So even if we know what rate our peer is supposed to honor, we still can't know if they are honoring it.

More generally, not honoring a promised rate is just one of many ways a connector can fail. A connector can also fail by dropping packets entirely or providing not very attractive prices. In order to test for any of these issues, we already need the kind of monitoring that you called "test payments". So if it were possible to remove the need for test payments in relation to connectors honoring quotes (as explained above, it isn't) we'd still need them for those other reasons.

TL;DR:

• Quotes as "meaningful promises" creates a free option problem, especially when combined with ILPv1 large amounts and ILPv1 route liquidity curves (which are relatively long-lived.)
• Connectors cannot detect if other connectors are charging too much even when there is a public destination amount.
• There are other failure modes other than not honoring quotes that also require "test payments".

[michielbdejong]

Yes! Conclusion: we need test payments. :)