discv5: sub-protocol data transmission

[fjl]

This is a proposal for a discv5 protocol extension that supports transferring arbitrary sub-protocol data over an encrypted connection.

Motivation

The motivation for this protocol is putting Portal Network's uTP connections on a more solid foundation. At this time, uTP transfers in the Portal Network use discv5 TALKREQ messages as a uTP packet enclosure. There are some downsides to that.

• The discv5 packet frame + TALKREQ message add overhead of ~90 bytes per packet, and processing of message packets is relatively expensive. The discv5 wire protocol is not designed for high-throughput data transfer connections. It is designed to efficiently perform short request/response exchanges with many different nodes. It is for this reason that every message packet sent must carry enough information to start a handshake.

• TALKREQ is a request message and requires a response. discv5 messaging and the handshake are based on the assumption that every request message triggers at least one response. TALKREQ is defined to have exactly one TALKRESP response, and for good reason: talk exchanges are intended as an upgrade path into another wire protocol (like the HTTP/1.1 Upgrade header and there is no guarantee on the ordering of responses. Allowing multiple responses is being discussed, but would create additional complexity in implementation APIs. If no TALKRESP response is observed, it is undecidable whether the recipient of TALKREQ has failed to receive the request, has decided not to respond, or the response got lost. Leaving out the response also breaks security assumptions in the handshake if TALKREQ is the initial message in a discv5 session (key confirmation does not occur).

Proposal

Session Table

Implementations should keep a sub-protocol session table, containing session records. Sessions are identified by the IP address of the remote node and the ingress-id value. Inactive sessions are removed from the table as they time out. Suitable session timeouts depend on the sub-protocol.

A session record contains:

• ip, the IP address of the remote node
• ingress-id, used to locate the session
• ingress-key, used for decrypting received packets
• egress-id, used as the session ID value in sent packets
• egress-key and nonce-counter, used for encrypting sent packets
• the sub-protocol that initiated the session

Establishing a Session

It is expected that sessions will be established through an existing encrypted and authenticated channel, such as discv5 TALKREQ/TALKRESP. There is no in-band way to create a session.

We assume that the implementation provides a procedure newsession which derives keys and creates a new entry in the table. Keys and ID values are created as follows.

newsession(initiator-secret, recipient-secret, protocol-name)
    initiator-secret :: bytes16
    recipient-secret :: bytes16
    protocol-name :: bytes

    ikm    = initiator-secret || recipient-secret
    salt   = ""
    info   = "discv5 sub-protocol session" || protocol-name
    length = 48
    kdata  = HKDF(salt, ikm, info, length)

    initiator-key = kdata[0:16]
    recipient-key = kdata[16:32]
    initiator-id = kdata[32:40]
    recipient-id = kdata[40:48]

    When called on the initiator side:
      egress-id, egress-key = recipient-id, recipient-key
      ingress-id, ingress-key = initiator-id, initiator-key

    When called on the recipient side:
      egress-id, egress-key = initiator-id, initiator-key
      ingress-id, ingress-key = recipient-id, recipient-key

In order to establish a sub-protocol session, the initiator creates its initiator-secret using a secure random number generator. It sends an appropriate TALKREQ message containing initiator-secret and any other information necessary for requesting a sub-protocol connection.

If the recipient agrees with the creation of the connection, it generates the recipient-secret and calls newsession() to create a session. It then sends an affirmative TALKRESP message containing the recipient-key, and possibly other sub-protocol specific data.

When the initiator receives TALKRESP containing the recipient-secret, it also calls newsession() to create and store the session. At this point the session is established packets can be sent in both directions.

Note that the first sub-protocol packet must be sent by the session initiator, since the session recipient doesn't know if and when the TALKRESP message will arrive. This limitation can be inconvenient for sub-protocols using a request/response scheme where data is to be served by the session recipient immediately after establishment. The first sub-protocol packet can have an empty payload in this case, but it really must be sent to confirm validity of the session.

The listing below shows an example packet exchange where node A is the initiator and node B is the recipient.

A -> B  TALKREQ (... initiator-secret ...)
A <- B  TALKRESP (... recipient-secret ...)
A -> B  sub-protocol packet
A <- B  sub-protocol packet
A <- B  sub-protocol packet
...

Packets

Sub-protocol packets have a simple structure with total overhead of 36 bytes, including the GCM tag (which is a part of ciphertext).

packet = session-id || nonce || ciphertext
  session-id :: uint64
  nonce      :: uint96
  ciphertext :: bytes

To send a sub-protocol packet for an existing session, the session-id of the packet is assigned from the egress-id of the session. nonce is selected by incrementing the session's nonce-counter value. It is recommended to also fill a part of the nonce using a secure random number generator. Now the ciphertext is created:

ciphertext = aesgcm_encrypt(egress-key, nonce, payload, egress-id)

When the node receives a UDP packet, it first checks that the packet data has a length of at least 20 bytes. It then performs a lookup into the sub-protocol session table by interpreting the first 8 bytes of the packets as a session-id. This value is used to look for an active session with a matching ingress-id and IP address value.

If there is no matching session, the packet is considered off-protocol and is submitted for processing as a regular discv5 packet.

If a session exists, the node performs AES/GCM decryption/authentication. Packets failing this step are discarded. If authentication succeeds, the session's idle timer is extended and the decrypted plaintext is dispatched to the sub-protocol implementation.

Security Considerations

This section explores some of the design choices from a security point-of-view.

• The key agreement scheme assumes an existing encrypted and authenticated communication channel. As such, key material is passed directly between participants. Any breach of session keys for this channel is also a breach of sub-protocol session keys.

• Both parties contribute key material used for session identifiers and keys. This is done to ensure that plain-text packet data cannot be predetermined or assigned with malicious intent by the initiator or recipient. It's also convenient because only a single value needs to be communicated across during session establishment.

• Sessions can be created with little overhead. Implementations should place limits on the number of concurrent sessions that can be created. It is good practice to have a limit on the total number of active sessions, because an attacker could use a large number of nodes to work around address-based limits.

• Since session-id values are transmitted plain-text, an observer in a privileged network position will be able to determine which packets belong to a single session.

• The protocol does not provide any ordering or transfer reliability guarantees. Sub-protocols are expected to provide such guarantees if needed.

[pipermerriam]

Rough/loose internal plan from portal network side is to do a POC implementation of UTP on top of this protocol change so that we can get some hands-on experience with the ergonomics.

The only "concern" that I have is the plain-text session-id and the possibility of network level packet filtering, however, such filtering would only end up applying to a single session and its likely that we would do something like use a new session for each UTP stream... so I'm not really sure that it would even be very effective/feasible for someone to try and do this kind of filtering. Do you have any additional thoughts here? I assume that the overhead of making the packets fully opaque adds unwanted overhead to the packet and maybe was deemed un-justified given the limited effectiveness of this kind of filtering (since establishing a new session is reasonably trivial...).

[fjl]

Nice that you are considering to implement it! I personally haven't tried to implement it yet...

I think it's not possible to apply filtering to this protocol, for a single session or at all, because:

• Session IDs used by the are random numbers that can't be guessed ahead of time.
• Session/stream tagging is very common in all kinds of protocols. It's not like we're the only protocol that sends UDP packets where the first field is a number identifying the data stream.
• Everything else in the packet is random data. We have a dedicated obfuscation mechanism in discv5 because the packet frame has multiple identifying features, i.e. specific byte offsets that will always have the same value in every packet, but that's not the case for this protocol.

[emhane]

Why divide up the receiving and sending with two encryption keys? It doesn't clearly state here where the sub-protocol session management should be done. I think it should be done by the app running discv5 and packets should be passed directly through to the app as the tuple (src-addr, encrypted-packet), and down to the discv5 socket as (dst-addr, encrypted-packet). This means that encryption is a black box to discv5 and the app can tell discv5 to blacklist certain malicious peers if it so wishes based on failure to match a session or whatnot. What I think belongs to discv5 protocol of this is

• the packet decoding and encoding at socket level as (connection-id, nonce, encrypted-data), to enforce mechanisms to circumvent packet filtering so discv5 can guarantee delivery of packets to the app,
• key sharing via TALKREQ responded with TALKRESP. A TALKREQ is always answered with TALKRESP as not to disturb original discv5 flow. After that what goes inside should be a black box to discv5 and up to the app based on how it wishes to encrypt its sessions. It could be as simple as TALKREQ { let's use this key(s) } and TALKRESP { roger }.

How to do session management I think serves as a suggestion but doesn't belong in the discv5 protocol.

I started implementing this in rust here: https://github.com/emhane/discv5/tree/tunnel-discv5.2 and here https://github.com/emhane/tunnel

[fjl]

Just to clarify: the proposal is not intended to be a part of the discv5 wire protocol spec. It is just intended to show how sub-protocol sessions/multiplexing can be done at all, because doing this safely isn't entirely trivial.

The extent of discv5 integration here is: discv5 is the transport for session establishment, and both protocols can run on the same port.

[fjl]

Why divide up the receiving and sending with two encryption keys?

It is a common practice to do that. Having a separate write key on each side avoids issues where the nonce could be reused, among other things. TLS does this too. You can read more about it in this StackExchange answer. The discv5 wire protocol also uses a separate write key for each side.

[fjl]

I have created a working prototype implementation over at https://github.com/fjl/discv5-streams/tree/main/session.

[emhane]

I've been implementing this crypto so rust and go can interface. Under heading Packets, it's 22 bytes right? the tag is u16?

[fjl]

GCM tag size used in the Go code is 16 bytes (i.e. u128).

The packet header is 20 bytes (8 bytes session-id, 12 bytes nonce).

[fjl]

We could go with a smaller session-id. 4 bytes is probably sufficient. But let's try interop with ID size 8 bytes for now.

[emhane]

GCM tag size used in the Go code is 16 bytes (i.e. u128).

The packet header is 20 bytes (8 bytes session-id, 12 bytes nonce).

in rust too it's 16 bytes, my bad, got confused at the end yesterday with the aes-gcm crate's own type for unsigned ints. nice, yeah let's interop with 8 bytes id.

[emhane]

utp accounts for the unordered arrival of udp packets and we can make utp transmit an event to a session manager when a seq_num has already been seen so the session is failed, I'm expecting the code looks somewhat similar in go. so the nonce counter is purely to make sure that the packet is unique so the cipher can't be broken with analysis, correct?

[fjl]

Yes, the nonce of the outer packet frame is for encryption/authentication purposes only.