Closed vincenzopalazzo closed 1 month ago
How do folks feel about spending 1 if not 2 of the 3 days discussing some of the biggest fundamental challenges with the LN protocol as it is today and then potential long-term solutions?
Example fundamental challenges with LN:
Nothing would be off limits for solutions including long-proposed ideas such as eltoo/LN Symmetry, channel factories, and coinpools as well as more recent proposals such as sidepools and timeout trees. Protocol changes that require bitcoin consensus changes around covenants or GSR would be within scope of discussion.
Thoughts?
I completely agree with @moneyball's proposal and think it's worth spending 2 days on that topic. We're at a stage right now where we're all working on implementing a few large improvements for which the spec is somewhat final (dual-funding, splicing, taproot, bolt 12, liquidity ads). These features have been discussed at length already and simply need engineering effort to get to the finish line, without any significant blocker. It's a great time to brainstorm what happens next, after all of these features, and ensure that we're working towards a satisfying long-term network!
And we can keep 1 day to iron out the details of some of the short / medium term work.
From my perspective, there are a few points I'd like to discuss that can fit well inside 2h of the 3th day after we discuss the fundamental limitation of the current lightning, and also could help to improve the "engineering effort" required to implement features proposed like dual-funding, splicing, taproot, bolt 12, liquidity ads.
If anyone wants ~nightmares~ ideas for topics I tried to map out open problems here a while ago: https://miro.com/app/board/uXjVN6hnZj8=/
I'd also like to discuss our decision to use Delving Bitcoin and gather feedback on whether we should continue with it or consider alternative ML approaches.
Is there a reason we need to cover this at the summit? Seems like an easy enough topic for a regular meeting, we should just add it to the agenda sometime.
Unordered list:
I'd like to spend some time brainstorming our not-so-future use of TRUC transactions and 0-fee commitments, the simplifications it provides and the trade-offs it creates for wallets that thus need to keep on-chain funds available in case their LSP cheats.
I'd also like to discuss (quickly) a few current topics that we should be able to finally close:
Some items on my mind lately (mostly future stuff):
musig2
output to create a 1 level tree of channels. stfu
, before doing the final boss of the channel state machine. unadd
, which allows channels to actually recover from state transition violations/bugs. min(1 mSAT, max(route.min_htlcs))
and the other the actual payment. This ends up running into a two general's like problem: what if the ACK gets stuck...?@moneyball wrote:
Example fundamental challenges with LN:
- Onboarding costs for new users (due to on-chain costs and cost of capital)
- Small balances aren't economically feasible (due to on-chain costs)
- Regulatory/trust challenges with channel management for mobile users
- LN today is a step in the wrong direction to achieve p2p digital cash as it needs numerous cloud servers/services (block/txn data, LN state backup, RGS-like service for performant pathfinding, probing scoring data service, LSP/JIT channel partner, async payments channel partner)
Extending my last post on delvingbitcoin about the likelihood for payments to be infeasible I have recently published the current draft (first 10 pages are pretty stable) of my research entitled "A mathematical theory of payment channel networks". From this I'd like to add a 5th point to @moneyball's list:
The model and results of the linked research suggests that 2-party channels are too constraint to leave users enough flexibility to reliably conduct payments. The geometric properties of the model also explains why we observe so many depleted channels in practice and that this is to be expected and can hardly be mitigated through liquidity management (neither off-chain nor on-chain).
The positive side of those results indicate that the direction that @Roasbeef proposes seems to have more advantages than those mentioned by him:
Off chain channel on boarding:
A class of protocol extensions designed to enable more than one channel to exist within a single output. This includes ideas like channel factories, variants of ark, etc, etc.
Ark is a bit different, since all outputs expire after a period of time, so all channels need to be rolled over or the funds are forfeited. As a result, all channels created in an ark end up being ephemeral channels.
The goal here is to be able to reduce the last mile on boarding cost for new users, packing as many channels into a single output as possible.
I fully agree with the intuition that more channels per output are useful as we gain more flexibility for the liquidity without the necessity to conduct expensive on chain transactions. However at this time I belief we have sufficient evidence that the number of channels per output is not as important as the number of participants per output.
In the same repository I have a somewhat outdated notebook that shows that the more peers share an output (with all the downsides that are attached to it) the better the liquidity can be used by the participants of the network. This results in better payment flows and higher reliability. Please take this notebook carefully as there is still a methodological error included. In reality the results will be less optimistic. Yet I do belief that the trend of the following two diagrams from the notebook should still hold true even if the systematic error is fixed:
This shows rates of infeasible payments of a current lightning network snapshot for payments of 1M sats. The multi party cases are constructed by starting from the current topology of the 2-party channel graph and randomly adding existing peers to the channels.
The end points of the diagram when the curves have converged are put to this diagram:
This is rather strong evidence that multi party channel networks seem to have the additional advantage to reduce the rate of infeasible payments. As far as I know this property of multi party channel networks was not explicitly known / discussed before and I think it is useful to keep this in mind when discussing @Roasbeef's ideas / proposal.
Hybrid Reactive/Proactive Routing:
- Structurally, today we use a proactive routing protocol. All nodes have some version of the graph, and then use that for payments. Other than new links and policy updates, no other information is exchanged.
- Overtime, people started to probe more as a way to figure out if a route can even work in the first place, leading to a hybrid protocol wherein nodes proactively maintain the channel graph, but then at times reactively send out a probe to gauge current network conditions.
- Should we consider a more concrete cut out for experimentation with a better specified hybrid protocol? So something similar to old reservation systems people have proposed in the past, where a non-binding intent to route is sent out a feeler (no HTLC) which can be used to gain up to date information re internal network conditions.
This isn't without tradeoffs ofc:
- Non-binding so nodes can lie, or even overpromise (say yes to everything beyond your capacity).
- Enshrines probing as a thing, potential source of additional balance/privacy leak.
The afore mentioned research indicates that the rate of infeasible payments cannot be reduced through a better routing protocol. However a reduction in uncertainty and improvements in routing protocols can help us in the good case to deliver the payment more quickly and in the bad case to fail quicker. Thus I generally support this direction and will add the lowish hanging fruit of optionally sharing liquidity information within the friend of a friend network.
I'd like to discuss the BOLT12 Payment Notification proposal, which aims to extend BOLT12 to cover the LNURL-verify use case.
I'd like to discuss the BOLT12 Payment Notification proposal, which aims to extend BOLT12 to cover the LNURL-verify use case.
More broadly, we should discuss "next steps" for after the BOLT12 spec is merged. In addition to payment notifications, we've seen interest in requesting a refund given an invoice + preimage. And the notification proposal includes a proof-of-payment mechanism, so we may want to keep that in mind when thinking about this.
Forwardable peerswaps. Yes, it is not as cool as multiparty (N > 2) offchain cryptocurrency mechanisms, but it is effectively multiparty in that a single onchain transaction (potentially) affects multiple channels, can be deployed with less code than actual N > 2 cryptocurrency mechanisms, and is simpler even than batched splicing.
We've posted some results and updates to our proposal on delving for discussion at the summit. It's a long post (for a long flight perhaps!), but if you only read one thing please start reading at the sink attack section. We've tested the mitigation against various types of attacks, and would like to do a whiteboard session to refine the changes to the proposal we're looking at making, and discuss tradeoffs.
Spend some time discussing the V3 commitment format, and how we're going to go about upgrading (upgrade for anchors / upgrade for taproot / both?). Would be great to leave the summit with agreement on the new commitment format and a solid plan for upgrade mechanism.
I think it will be worthwhile to do a "shill, kill or merge" session to clean up the issues and PRs on the BOLTs repo. Can be done in a low-brainpower (/high jetlag) hour, where we run through everything that's open and either:
We're generally quite good at merging in our online meetings, so we'd mostly be killin and shillin 🔪
Would be great to discuss the SuperScalar proposal which is perhaps more actionable than John Law's timeout trees as it doesn't require a consensus change. Although one can argue this isn't appropriate for this dev meetup since it doesn't require a BOLT change as any LSP could implement this. (but, there still might be broad interest among this group as a way to address some of LN's drawbacks) https://delvingbitcoin.org/t/superscalar-laddered-timeout-tree-structured-decker-wattenhofer-factories/1143/5
I'd be interested in having some time dedicated to discussing LN symmetry and the covenant proposal landscape as it applies/impacts the ability to add symmetry.
More of an informational and status session on the current symmetry project and what list/set of updates onchain are needed to unlock it.
It was great to see everyone, thanks everyone for all the energy and ideas! And thanks for Vincenzo for organizing everything!
Sharing my notes from the topics I was involved with:
CSV 1
on outputs and CPFP from to_remote
or HTLC txs (endogenous fees FTW)interactive-tx
to set the transaction's versionsubmitpackage
RPC, which isn't available for electrum client (e.g. mobile wallets)update_fee
and use package relay to force-closeoption_simple_close
:
shutdown
: we don't need to re-send ours, can just send signatures immediately for the remote txeclair
and lnd
, should be ready sooncontact_id
in their invoice_request
if they wish to add the recipient to their contactscontact_id
s for the same contact pair)payer_id
field of the invoice_request
@rustyrussell Can I ask you a simple question in a cold and polite fashion ? As I think you're still the older active contributor to the lightning protocol so far.
If you could give to the general audience what were the technical criterias and how many people were invited to this Lightning Dev Summit. Not asking the names of the attendees, just if a formal process was followed in constituting the invitation list to ensure consistency, neutrality and bright lines.
In the past, you published such piece about the “Corruption of Ethics in Cryptocurrencies" and I must say it was a good one. Though more recently you had strong (hostile ?) comments on twitter (here) about the behavior of some other contributors, and now I’m very confused if you're not acting with double-standard of ethics, or more concerning deliberately doing delusional statements in this industry.
I don’t think there is no need to yell about your past experiences about the Linux kernel of decades ago. Used to do linux hacking and familiar how to debug a syscall table and more frankly, I don't think a lot of things are great about Linux culture.
Here, I guess I should finish by quoting some Aussie writers, philosophers or historians, as well people on the internet are driven by different cultural norms. But the only one I know is Samuel Alexander (through Whitehead) and I must confess I have not read it.
I can answer that because I originally invited people. The list was "people who actively contributed to the spec (or were invited by a major lightning implementation cause they regularly work on it) or were otherwise at the last one".
Also, closing this issue now since the summit is over. Thanks everyone for attending and thanks again to the folks who organized it!
Here's the raw notes I took during the summit (working on typing up a longer summary): https://docs.google.com/document/d/1erQfnZjjfRBSSwo_QWiKiCZP5UQ-MR53ZWs4zIAVcqs/edit?usp=sharing
LN Summit 2024 Topics
Tracking issue to collect the topics that folks are interested in discussing during the lightning dev summit in September.
Please submit topics that you would like to discuss in the comments.
General Guidelines:
The notes from last year's summit can be found here.