Reliable Stream?

[jbenet]

WebRTC has reliable streams, both TCP and SCTP/UDP. Currently go-natty only exposes a UDP

Does natty expose these? go-natty only exposes UDP. (i can see about adding TCP/SCTP to go-natty if natty does).

And-- if not -- how do you do reliable transport? I haven't found a solid reliable wrapper in Go around a UDPConn yet.

[jbenet]

@oxtoacart great idea: https://github.com/oxtoacart/go-udt - UDT is really cool. QUIC is also worth considering the long term (no handshake RTT burn), but in the meantime?

[myleshorton]

@jbenet If I understand the question in the meantime our intention is to use UDT. I don't think QUIC is quite ready for prime time, but we're on the same page in terms of it being promising and have been in pretty close contact with Google on it for awhile.

To step back for a second, our intention for all of this is actually to provide pluggable reliable-UDP transports without a NAT-traversed context. The idea is partially to experiment with different transports but more importantly to stay unblocked in censored regions. We currently deploy things like fteproxy (https://fteproxy.org), and will likely run that over a reliable UDP transport in the future. The transport itself will also likely become more dynamic and harder to fingerprint over time.

We don't deal with TCP in a WebRTC context or within Natty currently simply because NAT traversal over TCP works in a much lower percentage of cases and I'm not sure there are any NATs where TCP traversal works and UDP traversal doesn't.

[atavism]

@myleshorton From what I've read, WebRTC itself uses TCP depending on the situation. The ICE framework was extended with RFC https://tools.ietf.org/html/draft-ietf-mmusic-ice-tcp-16 to return TCP candidates so it must happen occasionally. We could modify natty to return all successful connections (or filter by transport type) rather than terminating on the best connection, even if TCP NAT traversals seldom succeed (maybe a use case would be behind a corporate network that only allows access to the Internet over TCP for instance?). After two peers are NAT-traversed, natty itself, instead of shutting down right away, could be configured to persist the connection, according to a command line flag perhaps? That means instead of being just a pure NAT traversal tool, it could also be used to expose generic WebRTC data channels, that could be made reliable or not (which would switch between SCTP/UDP) with another flag. Anyway, just some things worth mentioning since the flexibility/extensibility is there.

[myleshorton]

@jbenet Makes sense, although I'm actually not aware of WebRTC implementing ICE TCP, but I could be wrong. That's actually an RFC at this point -- https://tools.ietf.org/html/rfc6544.

I'd be curious to see if there are cases where TCP traversal succeeds and UDP doesn't but I know at least Skype will fall back to TCP in some scenarios, I believe typically with certain corporate firewall configs but again not sure.

With the whole pluggable transports concept we're ultimately open to any transports being plugged in, but we probably do want to keep it super modular and have Natty itself focus on purely NAT and firewall traversal while making it really easy to plug in SCTP, QUIC, UDT, Aspera's stuff (probably the best of all), TCP, whatever (with TCP of course depending on the type of traversal) on the transport side.

[oxtoacart]

+1 on keeping natty focused on NAT traversal

+1 on exposing the sctp transport from WebRTC if there's value there (but as a separate project). Note - I think it may just be using http://www.sctp.de or some variant thereof.

+1 on emitting TCP five tuples from natty if WebRTC happens to return them On Nov 16, 2014 10:42 PM, "myleshorton" notifications@github.com wrote:

@jbenet https://github.com/jbenet Makes sense, although I'm actually not aware of WebRTC implementing ICE TCP, but I could be wrong. That's actually an RFC at this point -- https://tools.ietf.org/html/rfc6544.

I'd be curious to see if there are cases where TCP traversal succeeds and UDP doesn't but I know at least Skype will fall back to TCP in some scenarios, I believe typically with certain corporate firewall configs but again not sure.

With the whole pluggable transports concept we're ultimately open to any transports being plugged in, but we probably do want to keep it super modular and have Natty itself focus on purely NAT and firewall traversal while making it really easy to plug in SCTP, QUIC, UDT, Aspera's stuff (probably the best of all), TCP, whatever (with TCP of course depending on the type of traversal) on the transport side.

— Reply to this email directly or view it on GitHub https://github.com/getlantern/go-natty/issues/4#issuecomment-63259147.

[jbenet]

I don't think QUIC is quite ready for prime time, but we're on the same page in terms of it being promising and have been in pretty close contact with Google on it for awhile.

What did they say? any idea of when it will at least reach a stable spec?

To step back for a second, our intention for all of this is actually to provide pluggable reliable-UDP transports without a NAT-traversed context.

Yep, awesome. It's two layers, really.

I'm not sure there are any NATs where TCP traversal works and UDP traversal doesn't.

I actually have seen many! These are common inside large networks. I've seen lots of corporate networks drop all non-dns UDP traffic. Also, some networks (including the fucking Stanford visitor network, which I'm relegated to after graduating) blocks all non :80 :443 target TCP ports too. In my view, only FTCP is guaranteed to work (in networks that purport some notion of connectivity).

We could modify natty to return all successful connections (or filter by transport type)

+1 !

After two peers are NAT-traversed, natty itself, instead of shutting down right away, could be configured to persist the connection, according to a command line flag perhaps? That means instead of being just a pure NAT traversal tool, it could also be used to expose generic WebRTC data channels, that could be made reliable or not (which would switch between SCTP/UDP) with another flag.

I agree with @oxtoacart -- i think this should be a separate project. (and maybe have a third that bundles them, but really want to provide the slimest thing you can).

I'd be curious to see if there are cases where TCP traversal succeeds and UDP doesn't but I know at least Skype will fall back to TCP in some scenarios, I believe typically with certain corporate firewall configs but again not sure.

Not only that, but two peers are not guaranteed to be able to connect. You might have to use relays. I'm sure you know this (given lantern is all about relays :) ), but for others reading, what you need to do is provide some server to relay the connection (this is one reason Skype was very successful, it helped relay tons of traffic from NATed nodes over publicly accessible peers).

With the whole pluggable transports concept we're ultimately open to any transports being plugged in, but we probably do want to keep it super modular and have Natty itself focus on purely NAT and firewall traversal while making it really easy to plug in SCTP, QUIC, UDT, Aspera's stuff (probably the best of all), TCP, whatever (with TCP of course depending on the type of traversal) on the transport side.

+1.

---

@oxtoacart (or whoever did the natty embedding) can we chat briefly about the best way to bundle UDT? I think you have more experience than me on this. you can find me at #ipfs on freenode. (everyone's welcome to come hangout!)

[jbenet]

And, I think it shouldn't be hard to make go bindings for http://www.sctp.de/sctp.html -- I'll look into it after we find a good way to bundle UDT.

[oxtoacart]

Assuming he's available, @atavism and I will try to find you on irc sometime. On Nov 17, 2014 8:08 AM, "Juan Batiz-Benet" notifications@github.com wrote:

And, I think it shouldn't be hard to make go bindings for http://www.sctp.de/sctp.html -- I'll look into it after we find a good way to bundle UDT.

— Reply to this email directly or view it on GitHub https://github.com/getlantern/go-natty/issues/4#issuecomment-63327731.

[myleshorton]

I don't think QUIC is quite ready for prime time, but we're on the same page in terms of it being promising and have been in pretty close contact with Google on it for awhile. What did they say? any idea of when it will at least reach a stable spec?

We actually haven't chatted with them (the WebRTC team really) about it for probably a year, and it looks like it's making steady progress:

https://chromium.googlesource.com/chromium/src/net/+log/master/quic

It definitely has the nice advantage that it's being very actively developed, so maybe it is really the most promising of the bunch?

I actually have seen many! These are common inside large networks. I've seen lots of corporate networks drop all non-dns UDP traffic. Also, some networks (including the fucking Stanford visitor network, which I'm relegated to after graduating) blocks all non :80 :443 target TCP ports too. In my view, only FTCP is guaranteed to work (in networks that purport some notion of connectivity).

Interesting. So the Stanford network will allow incoming TCP on :80 and :443? I've certainly seen a lot of networks only allow them for outgoing but not incoming. Out of curiosity, does the Stanford network give everyone public IPs?

Not only that, but two peers are not guaranteed to be able to connect. You might have to use relays. I'm sure you know this (given lantern is all about relays :) ), but for others reading, what you need to do is provide some server to relay the connection (this is one reason Skype was very successful, it helped relay tons of traffic from NATed nodes over publicly accessible peers).

Right. In Lantern's case we actually don't touch relaying at all because anywhere a node might theoretically relay we just proxy directly (because in our case we'd just be relaying to a proxy). So that's a little bit unique because our goal/need is not necessarily to connect two specific peers but rather to ultimately access blocked sites.

[jbenet]

So the Stanford network will allow incoming TCP on :80 and :443? I've certainly seen a lot of networks only allow them for outgoing but not incoming.

No, i mean they only allow TCP 80/443 OUTgoing. it's absurd, tons of websites break (cause they use services exposed at other tcp ports. Note this only the visitor network. The actual network (which they lockdown administratively) is awesome, incredibly fast, and not very limiter. They do tons of research, so it has to be very open, provided you have proper access.

Out of curiosity, does the Stanford network give everyone public IPs?

Not to visitors. Only if you're working there, and request it. (Also stanford vs cs-dept at stanford are two different networks).

our goal/need is not necessarily to connect two specific peers but rather to ultimately access blocked sites.

You might want to do experiment with a real network on top of your point-to-point overlay then. (i.e. IP tunneled over a UDP flow).

[myleshorton]

I'm not sure there are any NATs where TCP traversal works and UDP traversal doesn't. I actually have seen many! These are common inside large networks. I've seen lots of corporate networks drop all non-dns UDP traffic. Also, some networks (including the fucking Stanford visitor network, which I'm relegated to after graduating) blocks all non :80 :443 target TCP ports too. In my view, only FTCP is guaranteed to work (in networks that purport some notion of connectivity).

Actually, so just to make sure we're on the same page on this, you're saying there are a lot of cases where UDP traversal fails due to aggressive firewalls but where TCP "traversal" succeeds with TURN correct? At least that seems to be true in the typical case, although there might be the oddball network that is surprising permissive with TCP even while blocking all UDP.

[jbenet]

there are a lot of cases where UDP traversal fails due to aggressive firewalls but where TCP "traversal" succeeds with TURN

Yes, but not necessarily with TURN. (i.e. yes in some cases you need TURN or any other relaying of traffic, non-transport specific, but there are many firewalls that just block all UDP and can actually be TCP holepunched.

These two things are sort of orthogonal (though correlated in "tight-fisted" net admin). UDP blocking is mostly firewalls, while NATs vary in their port assignment rules. (not sure what the distribution of symmetric nats is, but it's far from insignificant last i got a sense)

[atavism]

Another advantage of possibly using QUIC is that we're already working with the Chromium source (as a consequence of natty being based on WebRTC) so it wouldn't be too much work making use of that library with natty. I'd definitely be interested looking into exposing that directly, if we're open to that.

We actually haven't chatted with them (the WebRTC team really) about it for probably a year, and it looks like it's making steady progress: https://chromium.googlesource.com/chromium/src/net/+log/master/quic It definitely has the nice advantage that it's being very actively developed, so maybe it is really the most promising of the bunch?