RFC: Fake TCP with UDP

[jbenet]

In p2p systems, it's useful to use udp, as connection state and reliability can be done at a higher layer (i.e. {uTP,DCCP,SCTP}/UDP/IP). NAT Traversal sucks. Many aggressive NATs will block all UDP access. TURN sometimes fails. WebRTC has not gotten 100% connectivity.

As @feross pointed out, SSL/TCP is usually accepted. It would be pretty bad if NATs blanket-blocked HTTPS access. It is generally annoying to do p2p relaying on TCP, as TCP has all the session magic in its stack. It would be nicer to be able to use standard UDP (particularly when you have your own reliability / congestion control on top).

So, what if one does {uTP,DCCP,SCTP}/SSL/UTCP, Unreliable TCP. Use what -- to the NAT -- looks like legitimate TCP traffic, but is simply UDP. Will still have to keep track of seqnos (windowing) on a per-flow basis, as NATs/Firewalls may rewrite/drop a packet based on those.

 0                   1                   2                   3      
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1  
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+      
   |          Source Port          |       Destination Port        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Sequence Number                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Acknowledgment Number                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Data |           |U|A|P|R|S|F|                               |
   | Offset| Reserved  |R|C|S|S|Y|I|            Window             |
   |       |           |G|K|H|T|N|N|                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Checksum            |         Urgent Pointer        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Options                    |    Padding    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             data                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                            TCP Header Format 

Nothing else seems to matter. I think this would totally bypass most NATs.

[jbenet]

@feross @msparks @ali01 @davidad

[feross]

This is clever and would probably work!

[jbenet]

@cleichner

[davidad]

I think this is something that could only be determined experimentally. If it does work, the hardest part would be bypassing the OS TCP stack. This is, of course, possible—just annoying.

[jbenet]

@davidad yeah. raw ip in user land gets into permissions issues.

[msparks]

Agree with @davidad.

Do you have numbers on the percentage of "aggressive NATs" in the wild? Are they increasing in number or decreasing? UTCP, as clever as it is, may just be yak shaving for the last few percent of connectivity.

[jbenet]

In https://www.usenix.org/conference/nsdi12/technical-sessions/presentation/raiciu talks about real life middlebox problems. (it's also a good talk :) )

If +hangout users get "cannot connect to this other user at all" even 0.1% of the time, it would be unacceptable. In the end, need to relay any pair of connections. Ensuring any two hosts are capable of establishing a stream is not yak shaving IMO, it's basic internet functionality.

[davidad]

I think @msparks means that it's yak shaving vs. using regular TCP as a fallback, even if UTCP is "nicer". UTCP is still not guaranteed to work, so you'd probably need a regular-TCP fallback either way, and adding UTCP into the mix seems like a low payoff/complexity ratio.

[tv42]

Sounds like TCP-Minion.

http://arxiv.org/abs/1103.0463 http://dedis.cs.yale.edu/2009/tng/papers/nsdi12-abs https://groups.google.com/a/chromium.org/forum/#!topic/proto-quic/ROdV3ep2c8M http://tools.ietf.org/html/draft-iyengar-minion-concept-01 http://tools.ietf.org/html/draft-iyengar-minion-protocol-01

[jbenet]

Oh Minion looks pretty good

[lepasserby]

This is absolutely awesome (especially if one uses TLS to obscure actual data being transmitted) however, wouldn't that require kernel modifications (and thus become rather problematic for platforms like android, where building/loading your own kernel modules is still a god-damn adventure) ?

And yes, reminiscent of Minion, as well as this https://www.usenix.org/conference/foci13/workshop-program/presentation/nowlan Both minion and "uTCP" seem to require kernel modification tho :(

It would be extremely awesome if there was a way to make this work in a manner that (1) is not distinguishable from a "healthy" HTTPS/TLS session on port 443 for a competent, hostile observer (think China's Great Firewal thingie) (2) does not require kernel modification on user's side (thus allowing it to be useful for mobile platforms with limited/complicated kernel modification opportunities) (3) supports bi-directional unordered delivery (because "server -> client" head-of-line blocking is a bad thing even when "client -> server" head-of-line blocking is not present)

P.S.: Since we want complete indistinguishability from normal TCP, we should probably still ACK (otherwise a smart firewall will be able to tell something very fishy is going on) thus some retransmit overhead is unavoidable, but with out-of-order support being in place, that's probably not a terrible thing.

[tv42]

@lepasserby Kernel changes are for performance. Though injecting/receiving packets that look like TCP would require root privileges -> no unrooted Android etc. This is why piggybacking on UDP is so popular.

[lepasserby]

@tv42 Well, root is much less of an issue (absolute majority of android phones are rootable). Writing (and loading) a kernel module for android would be a PITA (and given the unhealthy diversity of android phones, compatibility will be an issue)

How severe would be perfomance degradation that one can expect without kernel changes (ballpark estimate) ?

[lepasserby]

Hmmmm... just a thought - it seems to me that android does in fact allow raw IP sockets for root apps. One could mayhaps establish raw IP connection and use it to transmit and recieve packets with headers that "look like" good TCP-TLS, but in fact are "FakeTCP-TLS/uTCP-TLS" ?

The server, most likely, will have a patched kernel to ensure "properly unordered" behavior.

The overhead, in terms of bandwidth wasted, would be [(retransmit overhead, since we have to follow retransmit behaviors even if we have unordered delivery) + (header-related overhead, since we have to feign TCP)]. Seems nice.

Would that work?

[wangyu-]

i have a repo implemented the similiar idea，use raw socket to send udp packet with fake tcp/icmp headers ，its basically a tunnel which supports almost all udp programs. https://github.com/wangyu-/udp2raw-tunnel