Closed fjl closed 3 years ago
@fjl, finally managed to read. Proposal 1 seems relatively sound imo, however I am not yet certain if Proposal 2 is necessary. I need to think about this a little further, don't quote me on this but potentially the same guarantees could be achieved if the transport of packets was done over quic. Quic it already ensures that middleboxes can't inspect packets, additionally this may be more sound as it would be using a transport google is already using for most of its webservices making it potentially even harder for middleboxes to block. I may be wrong though, I am no expert on the matter.
Quic it already ensures that middleboxes can't inspect packets
What QUIC is trying to prevent there is something else though. With TCP, all control information is transmitted in the clear. There are certain networking appliances which can optimize TCP traffic or apply advanced QoS rules by changing the parameters of a TCP connection passing through. QUIC makes this type of optimization impossible by design. It's not concerned with blocking.
additionally this may be more sound as it would be using a transport google is already using for most of its webservices making it potentially even harder for middleboxes to block
I can see the potential benefit of masquerading as a web protocol, but also think it's going to be quite hard to make the protocol truly look like it's a legit QUIC connection.
Here are some benchmark results for proposal 1 vs. proposal 2. These are from this go code on a 2,5 GHz Intel Core i7.
DecodeHandshakePingSecp256k1
is the handshake worst-case including ENR signature verification. DecodePing
is decryption & decoding of a ping packet with session keys.
name old time/op new time/op delta
V5_DecodeHandshakePingSecp256k1-8 526µs ± 0% 528µs ± 0% ~ (p=1.000 n=1+1)
V5_DecodePing-8 2.04µs ± 0% 3.51µs ± 0% ~ (p=1.000 n=1+1)
I think it's fair to say the additional masking crypto makes no difference at all.
@fjl Did header mask add any extra size to the packet or we have only IV packet size increase (16 bytes)?
The size increase is the added 16 bytes IV.
Just did another round of benchmarks on my phone, which has a Snapdragon 660 processor and sits roughly in the middle when it comes to smartphone processor performance.
V5_DecodeHandshakePingSecp256k1-6 1.29ms ± 1% 1.29ms ± 1% ~ (p=1.000 n=3+3)
V5_DecodePing-6 12.2µs ± 0% 19.6µs ± 4% ~ (p=0.100 n=3+3)
There is a possibility to get rid of crc64
computation that becomes redundant if both proposals are applied. Since dest-node-id
is used to derive a secret key, zero string can be used instead of crc64
as a key checksum value. This type of checksum should be pretty secure to use in this particular case as there is no risk to compromise the key.
Probably, we can also reduce the size of checksum from 64
to 32
bits. Assuming IV
is randomly generated (so does the secret key derived from randomly generated dest-node-id
) we can think of 2**32
randomly distributed checksums which is pretty enough to cover all nodes in all networks ever. Even though there is a collision, its cost should be negligible.
UPD:
IV
, read checksum (first 32 or 64 bits) check if they are equal to zero and either discard the packet or continue processing itThanks for the feedback. In the spec update (#157) the checksum is already removed.
This issue is a proposal for changing the discv5 packet format. I have included a lot of background information so everyone will be able to participate in the discussion.
There are a couple of requirements that drive the packet format design:
Why enable protocol identification for active protocol participants?
Protocol identification means that the application receiving the packet can distinguish discv5 packets from other UDP packets arriving at the same UDP port. This is different from DPI because the application understands the protocol and may have additional information which the firewall doesn't have.
Not all protocols include explicit identification. In fact, most Internet protocols don't because the port number already identifies the application sufficiently.
For peer-to-peer communication, protocol identification is useful because getting the traffic through NAT is hard enough to do for one port. Best if you can get the most use out of that port. Being able to confirm the protocol also enables deployment of new protocol versions alongside the existing version. For example, since discv4 packets can be matched by checking for a valid packet hash, we can run discv4 and discv5 on the same UDP port. Any traffic that isn't explicitly identified as discv4 can be passed on to the discv5 packet handler.
Why attempt DPI resistance?
I think it's important to keep DPI in mind when creating peer-to-peer protocols. Commercial-grade networking equipment provides built-in features for blocking peer-to-peer traffic (see Cisco, Meraki, Juniper docs), specifically file sharing protocols, because organizations don't want to expose themselves to legal risk. Most peer-to-peer networking activity is not illegal, but if it's easy to block peer-to-peer traffic for networking admins, they'll probably just do it to feel safer.
There are several ways in which a DPI-based firewall can identify traffic:
The main worry with DPI isn't that someone would block discv5 explicitly, it's rather that the protocol might eventually end up in a DPI vendor signature list. The discovery network can be a shared resource for multiple protocols. If a single application is deemed malicious and is blocked using DPI for that reason, all other non-malicious protocols will also be affected.
Designing the protocol wire format to avoid static signature matching is the simplest and most effective thing to evade most DPI. Working against timing/size analysis directly in the wire protocol is more complicated, but implementations can make themselves harder to identify by adding randomized delays or packet padding.
Note that DPI evasion measures do not make the protocol impossible to block in general, it just means DPI alone isn't enough to block it. It's still possible to block (or whitelist) the traffic based on port numbers, for example. Truly determined firewall operators could also just participate in the protocol and learn about node endpoints this way.
Protocol identification in the current discv5 wire protocol
The current discv5 wire protocol does not permit protocol identification explicitly. Worse yet, there is also no way to identify if an otherwise valid discovery packet is truly intended for the node which is receiving it. This causes the type confusion issue described in issue #131:
If node A sends a packet to an endpoint, assuming it belongs to node B, it uses a
tag
ofA xor sha256(B)
. But if the node behind the endpoint is actually node C, the protocol fails. Node C receives the message and derivessrc-id = (A xor sha256(B)) xor sha256(C)
which is bogus. It then sends WHOAREYOU back to the derivedsrc-id
, which node A doesn't recognize because the tag on WHOAREYOU depends on the destination ID.DPI resistance of the current discv5 wire protocol
While DPI resistance was a design goal initially, we sort-of abandoned it later when the handshake was added. I just re-checked it and turns out static matching is possible because the protocol contains plain text RLP.
WHOAREYOU packets with empty sequence number can be matched like this:
Another possible static signature is the authentication response header:
If either of these functions were used for blocking, discovery is effectively disabled. The selectors are not perfect and could be improved to reduce the false positive rate, but as an example, they show that static identification is possible.
Proposal Overview
For reference, the current outermost packet encoding is:
where
tag
is a multi-purpose 32-byte value,auth-header
is plain text RLP of varying size depending on the handshake state, andmessage
is an encrypted container for the actual protocol message. This encoding is very compact, in some cases even optimally compact, but has a number of real-world shortcomings:tag
construction and it doesn't guard against misidentification of nodes or the WHOAREYOU packet.auth-header
is multi-purpose and tries to hide the node record from passive observers in a clumsy way. During the handshake, some parts ofauth-header
are strongly encrypted with a one-time use key. This leads to a lot of complexity in the spec.The current encoding honestly just feels a little cobbled together. Given all those issues, I have decided to redo the outer packet encoding one last time, in a more principled way, before releasing the first stable protocol spec.
There are two proposals here. The first proposal defines a packet encoding using mostly fixed-size fields. The new format permits protocol identification explicitly (through the
checksum
) and encodes handshake state explicitly (inflag
). The second proposal builds on the first and adds 'DPI blinding', effectively removing all plaintext for passive observers.Proposal 1
The discv5 protocol deals with three distinct kinds of packets:
In the following definitions, we assume that the sender of a packet has knowledge of its own 256bit node ID (
src-id
) and the node ID of the packet destination (dest-id
). When sending any packet except WHOAREYOU, the sender also generates a unique 96-bitnonce
value.All packets start with a fixed-size
header
, followed by a variable-lengthauthdata
section, followed by the encrypted/authenticatedmessage
.The
checksum
field should be recomputed by the recipient based on its own node ID. The recipient may then verify whether the packet is truly a discv5 packet sent to the correct node. If thechecksum
doesn't match, the recipient should simply ignore the packet.The
flag
field identifies the kind of packet and determinesauthdata
content.Ordinary Message Packet (
flag = 0
)For message packets, the
authdata
section is just the 96-bit AES/GCM nonce:WHOAREYOU Packet (
flag = 1
)In WHOAREYOU packets, the
authdata
section contains information for the verification procedure.Handshake Message Packet (
flag = 2
)For handshake message packets, the
authdata
section has variable size since public key and signature sizes depend on the ENR identity scheme. For the "v4" identity scheme, we assume 64-byte signature size and 33 bytes of (compressed) public key size.authdata
starts with a fixed-sizeauthdata-head
component, followed by the ID signature, ephemeral public key and optional node record. Therecord
field may be omitted if theenr-seq
of WHOAREYOU is recent enough, i.e. when it matches the current sequence number of the sending node.Proposal 2 - DPI blinding
The encoding defined in the first proposal transmits all header information as plain text. This is fine, and does not affect the authentication property of the protocol in any way.
However, it may permit passive observers of discovery traffic to identify the protocol and uncover the node IDs which are communicating. Since the sender of a packet knows the destination node ID, and every node is aware of its own node ID, we can use the destination node ID as a symmetric encryption key for protocol metadata.
There are downsides to this obfuscation step: protocol debugging with standard tools (i.e. tcpdump) becomes impossible, and the additional
iv
element adds 16 bytes of packet size. We should carefully consider whether DPI resistance is worth enough to warrant the additional complexity and packet size.The image below shows the
masked-header
with a thick black border. Note thatmessage
is not part ofmasked-header
since it is already encrypted using AES/GCM.Decrypting the masked header data works as follows: The recipient first reads the
iv
and constructs an AES/CTR stream cipher using its own node ID as the key andiv
as the initialization vector. It can then decrypt theheader
part, verify the checksum, readauthdata-size
, and finally read the remainingauthdata
.