Provide some way to hook unix networking interfaces, for integrating legacy apps with ipNetwork and friends.

[zenhack]

@griff brought this up on the office hours call last night; it'd be nice if there was an easy way to adapt legacy apps that want to talk to the network using socket APIs, without having to make a bunch of invasive changes to those applications. For HTTP specifically we have sandstorm-http-bridge, and via HTTP proxies it's possible to build stuff like powerbox-http-proxy for porting legacy apps, but we don't really have a great story for this for apps that want to use other networking protocols.

Possible mechanisms we could use:

• @griff floated the idea of providing a tap device in the grain that such an adapter daemon could use to manage traffic. Before adding that I'd want to do a review of any historical security vulnerabilities around tap devices, to see if that would be too much attack surface to add.
• Historically I believe we had some iptables rules set up in the grain that would redirect traffic to some port? This was removed for some reason, I don't remember why. IIRC there were some limitations on what it was able to filter, but I can't remember the details. Should dig through the history and review this.
• Another idea i had is: If we assume most apps are going to be connecting via hostname, we could do something like the following:
  • Provide a way for a daemon inside the grain to handle DNS requests. Maybe we could bind port 53 ahead of time and pass the FD into the grain or something.
  • Create a dummy networking interface inside the grain, not connected to anything, and bind a large private address range to that interface.
  • The DNS daemon could then track connections to different hosts by assigning each of them a separate IP, and then listen for those connections and correlate based on IP. I think you could build most interesting things on this, and it has the advantage of not meaningfully increasing attack surface.

It would be useful to outline some concrete use cases for this and make sketch what a solution would look like for each of those. @griff, were there concrete applications you were interested in?

[kentonv]

@jparyani tried to do tricks with iptables for a while and had some limited success but also some headaches. But that was all five or so years ago and I doubt he or I could totally remember what the issues were. You might be able to find his app code in one of his github repos?

[abliss]

The DNS trick is a cute idea. But wouldn't the daemon have to listen on every single port? Is that even possible?

[zenhack]

I mean, depends on what you actually want to do? This might not be sufficient to implement a tcp/udp analogue of powerbox-http-proxy. But for many applications you'd have some knowledge and/or control of what ports the application is going to use anyway.

...though, it occurs to me that the DNS solution may run into the problem of not being usable if you need to connect to a low port#.

Probably we should pin down some concrete use cases for this before getting too deep into mechanism.

@griff, I may have asked this, but are there specific applications you had in mind for this?

Quoting Adam Bliss (2020-08-08 20:16:20)

The DNS trick is a cute idea. But wouldn't the daemon have to listen on every single port? Is that even possible?

[abliss]

Yeah, if you knew in advance which ports would be needed, this sounds more tractable. We could probably rig up authbind or something like it to handle low port numbers.

[abliss]

Or maybe it would be easier for the sandstorm server to expose a socks5 proxy interface inside the grain, and let apps choose one of these proxifiers ? https://web.archive.org/web/20171020003229/https://en.wikipedia.org/wiki/Comparison_of_proxifiers

[griff]

@zenhack in the short term I want to get SSH access to Googs and MQTT access to OwnTracks but those two only need IpInterface and are pretty easy to make a proxy for. In the the long term I have a secret dream of running all the software I run on my Mail/Web server and homebuilt NAS in Sandstorm but that is a long ways off. Mostly I am interested in adding full access to the whole of the internet using IpNetwork because it is an issue that caught my imagination when I first learnt of Sandstorm I have been thinking about how to implement such a thing on and off for several years now.

From my research I have found 3 major ways of giving full access using IpNetwork. 1) Injecting a library that takes over the socks method calls. Not something I want to do. 2) Using iptables REDIRECT or TPROXY to redirect all traffic to a single ip & port combo where you run a server that proxies the traffic. Documentation for how to actually do this in full has not been easy to find. I have mostly come across handwaving and toy examples. From the link that @abliss provided I found redsocks which uses this approach and I learnt more from skimming the source of that then I ever knew. The the old experiments used REDIRECT and one of the problems with that is that it actually modifies the packets while TPROXY does not. 3) Using a TAP/TUN interface which requires that you have your own Ethernet/IP stack. I recently came across an IP stack in Rust that looked promising and from looking at it and its example I have a couple of ideas for how to do this which is why I mentioned that approach. As for the security of TAP/TUN I believe that the main reason it is restricted to root is that it allows packet injection attacks and so can mess with the rest of the network stack but unless there are bugs in network namespaces those injections can't do much if the TAP/TUN device sits alone in its own namespace.

@abliss as for implementing a socks5 proxy I don't think that would be worth it as that is itself a pretty complicated thing and the proxifiers themselves would need to use one of the above methods anyway and so we might as well look at how the other proxifiers archive their thing and copy that.

[ocdtrekkie]

I definitely think if we're going to have any apps with direct networking access that we consider supportable in the market, we really need to be requesting individual TCP or UDP ports, and we really need an admin approval flow for these.

[griff]

@ocdtrekkie I partly agree with you in that we need to differentiate between admin only apps and normal apps. Around the code there is a mention of an idea of having drivers (admin grains that provide services to normal grains) and it would be cool to have that functionality. But that doesn’t change that I would like to have some way of doing full network proxying for those admin-only/drivers.

[zenhack]

I agree; I don't think a-priori the goal should be "make full network access easier"; most apps should be narrower in scope. But I could imagine a tcp/udp analogue of the http proxy I just wrote for the ttrss port.

---

Fwiw, socks5 isn't that complicated, and I already have a library that does half of the work here:

https://github.com/zenhack/go.socks5

It doesn't support udp or listening for connections (as opposed to establishing them as a client). But that was mostly because I never ended up needing them; they wouldn't be an enormous undertaking to add.

But socks5 support is spottier than http proxy support, so I'd wait until we actually had a use case where it would really help.

---

My concern about TUN/TAP is not that the interface may be theoretically unsafe, but that, as an API not normally exposed to unprivileged users, it may have received less scrutiny from a security perspective, and thus be more likely to contain exploitable bugs. I'm not dead set against it, but I would want to very thoroughly research any history of vulnerabilities, and carefully asses how much attack surface we're really opening up here.

From searching around, it looks like actually accepting packets redirected with TPROXY requires setting a socket option that itself requires elevated privileges (as you then have to work at the raw IP level). Not sure about REDIRECT.

---

My inclination here is to suggest trying to port apps you're interested in, and seeing what problems actually crop up in practice; I'd hate to add functionality to sandstorm and later find out it doesn't do what we need anyway, or that we'd have been able to solve the problems we actually hit without increasing attack surface.

[abliss]

@abliss as for implementing a socks5 proxy I don't think that would be worth it as that is itself a pretty complicated thing and the proxifiers themselves would need to use one of the above methods anyway and so we might as well look at how the other proxifiers archive their thing and copy that.

True, it'd be simpler to connect to capnp directly and skip the socks5 proxy. The benefit of the proxy would be to allow each app to decide for itself how to weigh the tradeoffs of the different proxifier methods. (If socks5 support were as widespread as http proxy support, this would be a slam-dunk, but I guess it isn't anywhere near that well supported.)

[ocdtrekkie]

@griff The thing is, an app like OwnTracks should work without you being a server admin. But a user also shouldn't be able to open arbitrary ports on the Sandstorm server. So a user should be able to ask an admin for a given app to access a given port, and let the server admin approve it (and of course, if they are the admin, they can immediately approve it in-line). We should as much as possible not special case Sandstorm apps for server admins.

Drivers would require an admin, but crucially, drivers (or system grains, as Ian has labeled his attempt at one for web publishing), should really be more like add-ins that extend Sandstorm functionality for other apps to use. So like, ideally speaking, maybe an email server is a driver that handles everything coming into the server on port 25, but that server, which is installed by the admin, in turn allows any user to use apps that depend on the email driver being installed on the server.

Completely open IpNetwork is actually a feature I strongly feel should be deprecated once we have a suitable replacement. "Allow all network access" is a permission level that userspace apps really shouldn't have on any platform.

[ocdtrekkie]

@zenhack

But I could imagine a tcp/udp analogue of the http proxy I just wrote for the ttrss port.

This would require an approval flow though, if you weren't an admin, yes? Because we'd be talking about a grain taking a given port on the server persistently, right?

[griff]

@ocdtrekkie Again I agree with you. If I was to to seriously work on MQTT support for OwnTracks (or SSH access in Gogs for that matter) I would do it as two apps. One being the MQTT driver that would require admin access and provides a cap for other apps that want to talk using MQTT and the other being the normal OwnTracks that could be set up to use that driver. I might do a private experiment on my own setup where this is not the case to work with and better understand MQTT but would not ever put that on the general app market.

[zenhack]

@ocdtrekkie yeah, it would probably make more sense for output connections. Which I guess raises the question: If we make it possible to request the ability to connect to a specific remote host/tcp port, should that be admin-only too, or could that be something unprivileged users could grant? It seems similar in scope to the current http mechanism.