Feature: Receiving BOLT 12 payments

orbitalturtle commented 2 months ago

Feature Description

So LNDK now supports sending BOLT 12 payments pretty well. But what about receives? As we start to explore this, I wanted to jot down some high level thoughts and some various tasks that need to be done.

Note that receiving with LNDK requires LND to be of version 0.18.3. so that we can use the --blinded option when creating an invoice. (0.18.3 is currently a release candidate, but I assume there should be an official release of this soon.)

Subtasks

[ ] Preliminary step: Our fork of tonic_lnd needs to be updated to support LND v0.18.3
[ ] Create an offer using LDK. To set the path field, we should be able to use LNDK's create_reply_path function.
[ ] When we receive an invoice request, we need to implement the logic here for responding with an invoice. This portion is where things are a bit trickier. LND v0.18.3 allows for creating BOLT11 invoices with AddInvoice with the --blinded option set. In order to return an adequate LDK Invoice, is it as simple as decoding the payment request provided by AddInvoice, then feeding the generated path and payment hash into LDK's InvoiceBuilder? Or are there other considerations here?
[ ] Once these pieces are more or less working, it'd be good to add a create-offer command to the server/cli to allow users to create an offer.

Open questions

How do we produce a persistent ExpandedKey?

One consideration @jkczyz pointed out months back is we'll need a persistent ExpandedKey when generating an offer. This key is needed in the offer flow to 1) verify that the invoice request we get back actually corresponds to the offer we produced, 2) derive a transient signing key to sign the invoice to return to the payer. Currently LNDK generates a new ExpandedKey each time it starts up, which doesn't work for this. IIUC, if implemented this way, we'd need to generate a new offer every time LNDK restarts, for whatever reason, which isn't feasible.

It seems like the easiest option for this would be to try to use LND's API somehow, if possible, to produce a persistent secret key. LDK for example generates the ExpandedKey from the master node key. The problem is that LND's API doesn't give us access to secret keys directly in the API response. So, we would be talk to LND devs to see if a PR would be welcome to allow exposing a secret for this one use case... But since LND's signing API is designed not to do so, I imagine there would be some resistance.

If that's the case, the only other option I see is to try to explore how to add some of the same signing functionality that LDK uses to LND's API without exposing the secret key directly in the API response.

I'll plan to talk to LND devs about what our options are here. But if anyone can think of any other options that I'm missing, let me know.

Other notes

The integration tests are located here for testing the flow of things and if we can properly make a payment: https://github.com/lndk-org/lndk/tree/master/tests

When this project is further along, it'd also be good to test more complex payment scenarios. Making updates to BOLT12 playground for this could be potentially useful here: https://github.com/LN-Zap/bolt12-playground

orbitalturtle commented 2 months ago

In order to return an adequate LDK Invoice, is it as simple as decoding the payment request provided by AddInvoice, then feeding the generated path and payment hash into LDK's InvoiceBuilder? Or are there other considerations here?

Pinging @carlaKC and @dunxen in case you have any further insight into adding receives to LNDK -- particularly on the above question 🙏🏻

jkczyz commented 2 months ago

In order to return an adequate LDK Invoice, is it as simple as decoding the payment request provided by AddInvoice, then feeding the generated path and payment hash into LDK's InvoiceBuilder? Or are there other considerations here?

Pinging @carlaKC and @dunxen in case you have any further insight into adding receives to LNDK -- particularly on the above question 🙏🏻

This likely is sufficient though someone familiar with LND may need to confirm. Those blinded payment paths typically contain data necessary to claim the payment and any other data needed by the implementation.

For paths created by LDK, we include a payment secret and a payment context. The secret is used to derive the payment preimage to release. The context is used to determine the PaymentPurpose used in PaymentClaimable and PaymentClaimed events. But since LDK isn't used to claim the payment, that data isn't needed and those events won't be generated.

carlaKC commented 2 months ago

For paths created by LDK, we include a payment secret and a payment context. The secret is used to derive the payment preimage to release. The context is used to determine the PaymentPurpose used in PaymentClaimable and PaymentClaimed events. But since LDK isn't used to claim the payment, that data isn't needed and those events won't be generated.

I think that hold invoices could be useful here, because they allow the invoice lifecycle to be controlled via API:

Create hold invoice with chosen payment hash
Subcribe to invoice updates to get notified when htlcs arrive
Settle/cancel invoices accordingly, supplying preimage via API

I think that may allow you to do the preimage derivation step in LDK/LNDK, which gets around the problem iirc. That said, blinded paths aren't surfaced on the AddHoldInvoice api afaik, so you'd need a change there to support this flow. More generally, it looks like LND doesn't allow you to externally supply a blinded path - it takes the appraoch of supplying a config and generating it for you, which seems like it could be problematic here as well if we need LDK to generate the blinded path?

jkczyz commented 2 months ago

Hmm... I don't think we need LDK to generate the blinded paths or payment hash. LNDK doesn't use ChannelManager -- which is what claims the payment and generates payment-related events in LDK -- so it shouldn't matter what's in the invoice.

The trickier problem is signing the invoice with the transient key derived from the offer's metadata. LDK will do it for you so long as the same ExpandedKey is used. But it would need to be persisted to work across restarts.

AndySchroder commented 2 months ago

As you are sketching this out, I'd encourage you to read this post I wrote up:

https://delvingbitcoin.org/t/privately-sending-payments-while-offline-with-bolt12/1134/1

If there is a way to also push an invoice_request to the node as an RPC command (such as if the invoice_request was received through a local physical link) and not just through onion messages, that would open up some new possibilities.

AndySchroder commented 2 months ago

You mention above about using the AddHoldInvoice api call to do some magic with LND in LNDK. Not to confuse things even further with that comment, but it would be nice to also have the ability to have hold invoices issued in response to an invoice_request from an offer in LNDK. Then have an option to release or cancel the hold invoice when you are ready to.

AndySchroder commented 1 month ago

As you are sketching this out, I'd encourage you to read this post I wrote up:

https://delvingbitcoin.org/t/privately-sending-payments-while-offline-with-bolt12/1134/1

If there is a way to also push an invoice_request to the node as an RPC command (such as if the invoice_request was received through a local physical link) and not just through onion messages, that would open up some new possibilities.

In addition, this RPC command should return with the invoice so that it can also be transmitted back over a local physical link.

AndySchroder commented 1 month ago

Currently LNDK generates a new ExpandedKey each time it starts up, which doesn't work for this.

What is the reason for this?

AndySchroder commented 1 month ago

kannapoix commented 1 month ago

If we can use LND SignerService SignMessage API, it is simple solution here instead of using LND private key as a ExtendedPubkey in LDK. The latter has many obstacles as mentioned in the original issue. To use SignMesssage API, we need to specify which key to use(they call it KeyLocator. It is index for derivation path).

@jkczyz Is it possible to put data representing KeyLocator in metadata or blined path data? If we could, we could have LND sign an invoice with a key that is in the location specified by the KeyLocator created from the metadata info.

jkczyz commented 1 month ago

If we can use LND SignerService SignMessage API, it is simple solution here instead of using LND private key as a ExtendedPubkey in LDK. The latter has many obstacles as mentioned in the original issue. To use SignMesssage API, we need to specify which key to use(they call it KeyLocator. It is index for derivation path).

@jkczyz Is it possible to put data representing KeyLocator in metadata or blined path data? If we could, we could have LND sign an invoice with a key that is in the location specified by the KeyLocator created from the metadata info.

Yeah, it's possible... but note that LDK achieves statelessness by using data from the offer (plus an ExpandedKey and a nonce) to construct a key-pair for signing. By having LND sign, you would need to implement this statelessness yourself. LDK also uses this construction to authenticate messages sent on a blinded path, so you would need to handle that independently as well.

jkczyz commented 1 month ago

Sorry, I let this slip.

Currently LNDK generates a new ExpandedKey each time it starts up, which doesn't work for this.

What is the reason for this?

IIUC, LNDK currently doesn't have a way to persist the ExpandedKey. @orbitalturtle Can provide more insight though.

Related: https://lightningdevkit.org/blog/bolt12-has-arrived/#achieving-statelessness ?

Yeah, the ExpandedKey is used there to construct the metadata. Note that as of LDK 0.0.124 the metadata is now stored in the offer's blinded paths instead of directly.

lndk-org / lndk