Detecting Probe-resistant Proxies (NDSS 20)

[wkrp]

Detecting Probe-resistant Proxies Sergey Frolov, Jack Wampler, Eric Wustrow https://censorbib.nymity.ch/#Frolov2020a

This research finds weaknesses in proxy-resistant proxy protocols, like obfs4 and Shadowsocks, that make them more prone to detection by active probing than previously thought. These are protocols that require the client to prove knowledge of a secret before using the proxy. Despite the fact that probe-resistant proxy servers are designed not to respond to unauthorized clients, they may have characteristic timeouts or disconnection behaviors that distinguish them from non-proxies.

The evaluated protocols have in common that the server reads some number of bytes from the client, then checks the authentication on those bytes. A typical code pattern is the following:

client, _ := listener.Accept()
client.SetDeadline(30 * time.Second)
buf := make([]byte, 50)
_, err = io.ReadFull(client, buf)
if err != nil {
    client.Close()
    return
}
if !checkAuthentication(buf) {
    client.Close()
    return
}
// client is authorized, server may now respond

The server reads exactly 50 bytes from the client, then checks the client's credentials. If the client doesn't send 50 bytes before the timeout, the server closes the connection. If the credentials are bad, the server closes the connection. Consider what happens when a unauthorized client sends 49, 50, or 51 bytes.

• With 49 bytes, the server times out after 30 seconds and closes the connection with a FIN.
• With 50 bytes, the server closes the connection immediately with a FIN (io.ReadFull succeeds but checkAuthentication fails).
• With 51 bytes, the server closes the connection immediately, but with a RST, not a FIN.

Why a RST in the 51-byte case? It's a peculiarity of Linux: if a user-space process closes a socket without draining the kernel socket buffer, the kernel sends a RST instead of a FIN. Put together, these distinctive timeout and FIN/RST thresholds form a fingerprint of the probe-resistant protocol, despite the server never sending application data.

The authors evaluate six protocols: obfs4; Lampshade (used in Lantern); Shadowsocks (the Python implementation and the Outline implementation, both with AEAD ciphers); MTProto (used in Telegram); and obfuscated SSH (used in Psiphon). They test a pool of known proxies, as well as large number of endpoints derived from a random ZMap scan (1.5 million) and from a passive ISP tap (0.4 million). They send these endpoints a selection of probes of different lengths. From these they derive simple decision trees for identifying probe-resistant proxy servers. (Where the root of the tree is always "discard endpoints that send application data in response to any probe.")

The decision trees classify a few endpoints from the ZMap and ISP tap sets as proxies. In the case of obfuscated SSH, the authors confirmed with Psiphon developers that 7 of the 8 identified proxies actually were proxies operated by the developers. In some other cases, there is corroborating evidence that the endpoints really are proxies, even if not direct confirmation. By far the most difficult protocol to identify is MTProto, because it never times out and never closes the connection. The authors recommend this strategy for the best probe resistance: when a client fails to authenticate, just keep reading from it forever.

For the most part, the developers of the examined protocols have fixed the identified flaws, at least by continuing to read from the client and not closing the connection immediately when there's an authentication failure. They may still have a timeout instead of reading forever, but the server will have identical reactions to the three cases examined above.

Thanks to Sergey Frolov for commenting on a draft of this summary and providing the code sample.

[wkrp]

I believe these are the source code commits that aim to mitigate the vulnerabilities found in the paper:

• Obfuscated SSH: Implement "read forever" for OSSH
• obfs4: Alter tear down behavior to be less distinctive
• Outline: Probing resistance via timeout
• Lampshade: Add init message timeout to improve resistance to probing

MTProto already worked as the paper's authors recommend, and therefore didn't require a patch. As far as I am aware, there has not yet been any patch to shadowsocks-python.

[ghost]

there has not yet been any patch to shadowsocks-python

Just mention, it's lack of maintenance, nearly nobody use it now. Most user are using C/Go/Rust implementation.

[NullHypothesis]

We discussed this paper in our anti-censorship reading group on April 2. Here's a summary of our discussion:

• It's unlikely that the paper's data contains any obfs4 bridges. The handful of obfs4 bridges that the decision tree captured are probably false positives – the same is true for Lampshade and probably for most MTProto proxies.
• We were surprised that the data contains many (true positive) Psiphon users.
• Why were curious what their results would look like over UDP. Many UDP applications don't respond by default. What if obfs4 was using UDP instead of TCP?
• There may be other data sources that, when combined with the paper's datasets, may allow an attacker to narrow down the set of potential obfs4 bridges. For example, most obfs4 bridges expose an OR port, which an attacker can discover by port scanning an obfs4 bridge.

[gfw-report]

It appears that there are still many popular circumvention tools having the weaknesses demonstrated in this paper as of June 2021. Possible reasons include lack of maintenance or incomplete mitigation.

In this issue (https://github.com/XTLS/Xray-core/issues/625), we shared a trick to quickly spot the weakness. In short, one can send 1) a large chunk of invalid data and 2) a 1-byte invalid data to the listening port (12345 in this example) of any circumvention tool:

python3 -c "print('a' * 900, end='')" | nc -v localhost 12345
python3 -c "print('a' *  1, end='')" | nc -v localhost 12345

If the behaviors are different, one can then start with a binary search to find the thresholds. Alternatively, one can try using the prober-simulator to analyze the reactions of the circumvention tools in a more systematic way.

When using the trick, keep in mind that the reactions of the server may not be deterministic. One may want to quickly repeated the test by:

for i in {1..10}; do python3 -c "print('a' * 900, end='')" | nc -v localhost 12345 && return; done