FR: Allow subnet routing in tsnet

clarkmcc commented 1 year ago

What are you trying to do?

Up until recently we've been using tsnet in our application to allow subnet routing to local /32 IP addresses. We were able to get it to work with this change however after updating to the latest version of Tailscale, this no longer works. How can we re-enable subnet routing in ts-net?

How should we solve this?

No response

What is the impact of not solving this?

We probably don't have a lot of use for our subscription if we can't get subnet routing in tsnet working again.

Anything else?

No response

clarkmcc commented 1 year ago

DentonGentry commented 1 year ago

Duplicate of https://github.com/tailscale/tailscale/pull/4284

The linked PR modifies tsnet to remove the local termination of TCP flows and to set ProcessSubnets to true. Did this stop working when the workspace used to compile tsnet was updated to HEAD, or when other nodes on the tailnet were updated, or did it stop working in some other way?

I expect that older version of tsnet to continue to interoperate with new versions of the Tailscale app, we take backwards compatibility very seriously. The oldest tailscale node still in operation is a Linux system we operate which is running version 0.100-154. For a long time it was running an even earlier 0.90 version, but now all user-operated devices running older than 0.100 have been retired and we updated it.

clarkmcc commented 1 year ago

I think it was when tsnet was updated to HEAD. If I apply this patch to HEAD, I can't get subnet routing to work anymore. I'm sure the protocol is backwards compatible, it looks like the tsnet package has evolved quite a bit since we forked last year so I was suspecting that maybe the current implementation required more than just setting ProcessSubnets... but I don't know.

clarkmcc commented 1 year ago

I'm sure this isn't a top priority for you guys so I hate to push for a direction on this, just curious if in the short-term we should expect:

This to be supported natively in tsnet
Instructions on how to modify our fork to get this to work again
Nothing, in which case we'll start figuring out something else

bradfitz commented 1 year ago

(2) you probably want to modify tsnet where it does ns.GetTCPHandlerForFlow = s.getTCPHandlerForFlow or ns.GetUDPHandlerForFlow = s.getUDPHandlerForFlow ... that's where it conditionally decides to handle a flow through it.

clarkmcc commented 1 year ago

@bradfitz exactly what I needed, thanks. Would you accept a PR that makes this a configurable option in tsnet, or are there reasons behind why this isn't supported out of the box?

diff --git a/tsnet/tsnet.go b/tsnet/tsnet.go
index 444990f2..b89a5dc8 100644
--- a/tsnet/tsnet.go
+++ b/tsnet/tsnet.go
@@ -521,8 +521,7 @@ func (s *Server) start() (reterr error) {
        return fmt.Errorf("netstack.Create: %w", err)
    }
    ns.ProcessLocalIPs = true
-   ns.GetTCPHandlerForFlow = s.getTCPHandlerForFlow
-   ns.GetUDPHandlerForFlow = s.getUDPHandlerForFlow
+   ns.ProcessSubnets = true
    s.netstack = ns
    s.dialer.UseNetstackForIP = func(ip netip.Addr) bool {
        _, ok := eng.PeerForIP(ip)

DentonGentry commented 1 year ago

Would you accept a PR that makes this a configurable option in tsnet, or are there reasons behind why this isn't supported out of the box?

tsnet is intended to serve an application. I'm sure you have reasons for not wanting to run tailscaled as a subnet router, but I'm having trouble seeing that as a widespread need. In particular, tsnet will always function in netstack mode and never in kernel forwarding mode: https://tailscale.com/kb/1177/kernel-vs-userspace-routers/

In basically all cases for subnet routing, we'd want tailscaled to be used. I'm not especially enthused about adding a configurable knob which we wouldn't actually want to support.

DentonGentry commented 1 year ago

I think the immediate issue is resolved.

clarkmcc commented 9 months ago

We're still maintaining a fork that supports this which is just fine. This is on my radar again because I'm just syncing up with upstream.

In basically all cases for subnet routing, we'd want tailscaled to be used.

To answer the question why we can't use tailscaled:

We embed tsnet in our application which greatly simplifies deployment (we don't have to install or uninstall tailscaled along with our application)
We don't actually want tailscale running all the time, our application controls when tsnet is up and when it's down.
We don't want a tun device that is able to be created and then be able to access our network. We have ACLs configured, but not having the ability for the client to somehow dial into our tailnet is a nice additional comfort.

I understand this is an obscure use-case, but I wanted to document it for future reference.

vinhjaxt commented 5 months ago

+1 need this feature

aidansteele commented 4 months ago

For what it's worth, I came across this issue after spending way too much time trying to figure out why tsnet wasn't behaving in the same way as the Tailscale app.

I think it would be useful to at least have the tsnet docs indicate that it can't be used to access subnet routes (my use case is that I am using tsnet as a client)

whmountains commented 2 months ago

@clarkmcc is your fork public, by any chance?

clarkmcc commented 2 months ago

@whmountains yes.

replace tailscale.com => github.com/printtracker/tailscale v0.0.0-20240209172008-24902dded0c5

tailscale / tailscale