rfq: creation initial version of RFQ last-mile negotiation module

Roasbeef commented 10 months ago

Overview

This is a master tracking issue towards the design and implementation of the rfq module with tapd. The rqc module will implement the abstract RFQ invoice last-mile invoice negotiation protocol outlined in the tap bLIP: https://github.com/lightning/blips/pull/29. This section in particular outlines the current protocol flow draft: https://github.com/lightning/blips/blob/f60cfb7bdd6fa266f88b80f0f65a5d5541862ad6/blip-tap.md#rfq-negotiation.

We can track this independently of any of the refactoring in lnd needed to expose hooks to modify the tapscript tree in each output as this module is nearly 100% independent of any of that logic.

The bLIP defines 4 new messages: tap_rfq, tap_req_accept, and tap_rfq_reject. We can use lnd's support for the sending+receiving custom messages to have this information be routed to tapd. By intercepting and sending these messages, two tapd instances connected via an active p2p connection can exchange the information needed to agree upon a rate for sending/receiving payments.

When it comes down to actually sending/receiving these HTLCs, we're able to once again re-use the existing RPCs that lnd offers.

For receiving, we'll need to take the rfq_id value generated, map that to a rfq_scid then use that as the scid for any created invoices. A set of new RPCs that give a caller more control over the active/sent scid aliases will need to be created. Such RPCs and the invoice creation hook are out of the scope of this tracking issue.

For sending, the very fist onion hop needs to add in a new custom TLV to inform the first hop node how to translate the incoming HTLC amt to the outgoing HTLC amount. The forwarding node can catch such HTLCs with the existing HTLC interceptor APIs. The sending node will need a new modified API that allows them to specify a custom TLV value for the first hop. This can already be done with the BuiltRoute and SendToRoute API calls, but we'll want an easy way to just hook this into the normal SendPaymentV2 RPC calls. Tracking this unit of work is also outside of the scope of this tracking issue.

Implementation

At a high level, the implementation will have the following components (not exhaustive, names to be considered place holders):

A sub-system that handles the incoming rfq message stream. This will pass off the messages to the central control module. We'll call this the RfqStream.
Another sub-system that tracks the set of active accepted quotes. Similar to the forwarding node logic mentioned above, this will need to use the HtlcInterceptor logic to reject incoming HTLCs that don't match one of the quotes, or don't adhere to the quote exactly (under pays, etc). We'll call this the RfqOrderManager.
A negotiation sub-system, that takes incoming requests, and consults the PriceOracle for guidance on if it should be accepted or rejected. We may also want a hook before that that'll decide if things should even be forwarded or not. We'll call this the RfqNegotiator.

`RfqStream`

This module will use the SendCustomMessage and SubscribeCustomMessages RPC calls to funnel the RFQ messages (which use the custom message range) to the RfqNegotiator. It should maintain book keeping to keep track of the RfqNegotiator instance for all the active assets the daemon cares about at the time. It'll also need to properly map the requests to responses as well (see the rfq_id value in teh messages).

`RfqOrderManager`

This module will manage the accepted set of RFQ bundles. Using the HtlcInterceptor RPC call, it'll attempt to interecept any HTLCs that have an scid that matches the set of active rfq_scid. The set of active rfq_scid will need to be persisted to disk in a reliable manner. In addition to using the interceptor to reject HTLCs that match the rfq_scid but don't adhere to the accepted terms, we'll also need a background routine that removes expired quotes (all quotes expire after a period of time).

We'll also want a set of RPCs to introspect into this state, and potentially modify, or expire the set of accepted quotes early.

`RfqNegotiator`

This is the module that handles the actual negotiation. Given an offer (expressed in the tick unit, see the bLIP) it'll consult the PriceOracle for that given pair. For the PriceOracle we can either assume some set REST API to hit to obtain the price in a standard manner, or create a streaming RPC version of the PriceOracle. The operators of edge nodes would then use that RPC to feed in custom prices into their node. These RFQ messages are also forward-able, so operators may farm them off to various other sources to be able to take advantage of larger pools of liquidity.

dstadulis commented 8 months ago

TODOs

[ ] Dis-aggregate/Break out sub issues from OP's design
[x] PR for wire messages
[ ] Assess the need for message priority to LND
[ ] set custom SCID
[x] interceptor extension
[ ] Use TLV recordT

dstadulis commented 8 months ago

User Story

Discussed deliverables with ffranr a bit As a dev-ops engineer at a LSP provider I want to be able to spin up and down worker nodes which have a subset of my overall logical node's channels to interact with the RFQ manager in order to have a high-availability design which is robust to failure and operational maintenance

Toying around with the idea that the architecture of the rfq manager might be coordinating many lnd node instances in which specifying a node identifier with the SCID.

dstadulis commented 7 months ago

Negotiator subsystem (takes quote and ACK or NACK price) will need to extend order management system: needs persistence on

streamer and peer messages
- remember assets RPC end points need to be added

ffranr commented 7 months ago

Let's shape the price oracle service here: https://github.com/lightninglabs/taproot-assets/issues/777

lightninglabs / taproot-assets