search

openwrt / packages

Community maintained packages for OpenWrt. Documentation for submitting pull requests is in CONTRIBUTING.md
GNU General Public License v2.0
4.04k stars 3.5k forks source link

tailscale: Tailscale in OpenWrt is vulnerable #24741

Open marek22k opened 3 months ago

marek22k commented 3 months ago

Maintainer: @p-w-p @1715173329 @neheb @mochaaP @BKPepe @boretom Environment: OpenWrt 23.05.4 r24012-d8dd03c46f

Description: The tailscale version in OpenWrt is vulnerable because it is much too old. The following message appears in the Tailscale Dashboard:

Security update available

This machine is running a version with a known security vulnerability. It’s recommended to update to 1.70.0.
Follow these instructions to update this machine.

(It links to https://tailscale.com/kb/1067/update?tab=linux)

Workaround:

# tailscale update
BKPepe commented 3 months ago

The newer version of Tailscale, which includes a security fix, is only available in the daily snapshots of OpenWrt (https://github.com/openwrt/packages/commit/a5028f282dac0d87cddf5fe17fd870822710e0c7). This is because Tailscale frequently requires the latest versions of Golang (https://github.com/tailscale/tailscale/commit/b6153efb7dd58ea5bdd97be45fbfeb0d3bb506dc). In the stable version of OpenWrt 23.05, Golang 1.21 is used, and as an open-source project, we are unable to assist this issue.

We can propose the following solutions:

The developers of the commercial product Tailscale could offer their assistance by releasing OpenWrt packages for stable versions or by providing support and solutions for OpenWrt. Otherwise, Tailscale may be removed from OpenWrt. Additionally, they could release a new version of Tailscale with the security fix that supports Golang 1.21. It is important to note that Golang 1.21 is still a supported version (https://go.dev/doc/devel/release - see "Release policy".

Lastly, it should be mentioned that tagging multiple irrelevant people on GitHub tends to cause more harm than good.

BKPepe commented 3 months ago

In my last comment, I forgot to add link to this https://github.com/openwrt/packages/issues/24570#issuecomment-2226988444 from one of our users, who said that it is not very applicable to typical OpenWrt usage.

marek22k commented 3 months ago

The newer version of Tailscale, which includes a security fix, is only available in the daily snapshots of OpenWrt (a5028f2). This is because Tailscale frequently requires the latest versions of Golang (tailscale/tailscale@b6153ef). In the stable version of OpenWrt 23.05, Golang 1.21 is used, and as an open-source project, we are unable to assist this issue.

If I'm not mistaken, this could also be related to the reason why Tailscale is not in Debian. Tailscale seems to be made more for rolling release (which is neither Debian nor OpenWrt). As far as I remember, it was said that the Tailscale developers don't want to offer LTS support.

The developers of the commercial product Tailscale could offer their assistance by releasing OpenWrt packages for stable versions or by providing support and solutions for OpenWrt. Otherwise, Tailscale may be removed from OpenWrt. Additionally, they could release a new version of Tailscale with the security fix that supports Golang 1.21. It is important to note that Golang 1.21 is still a supported version (https://go.dev/doc/devel/release - see "Release policy".

I'm in favor of that too!

Lastly, it should be mentioned that tagging multiple irrelevant people on GitHub tends to cause more harm than good.

Sorry. I'm always unsure when to say or how far back I should go in the commit history - apparently I tend to exaggerate a bit.

BKPepe commented 3 months ago

If I'm not mistaken, this could also be related to the reason why Tailscale is not in Debian. Tailscale seems to be made more for rolling release (which is neither Debian nor OpenWrt). As far as I remember, it was said that the Tailscale developers don't want to offer LTS support.

I would prefer not to debate the open-source WireGuard, which is integrated into the Linux kernel, versus the commercial Tailscale, which is a basically paid service and releases updates more frequently. It is true that Tailscale is not included in GNU/Linux distributions, and therefore, we should consider removing this package from OpenWrt as it is gonna solve several issues, which were recently opened (https://github.com/openwrt/packages/issues/24712, https://github.com/openwrt/packages/issues/24570, https://github.com/openwrt/packages/issues/24415, https://github.com/openwrt/packages/issues/22003).

Based on these two sources, I think it should be clear that it is not possible to keep up with the upstream:

It is beyond our capacity and resources to maintain the latest version of Golang in stable releases, as this increases space requirements and necessitates compatibility with other packages that use Golang. If someone is willing to take on the responsibility, time, and motivation to maintain it, that would be acceptable, but it would contradict the purpose of stable releases.

brada4 commented 3 months ago

Could you cite the vulnerability and cite upstream commit that fixes it? Best if xrefd with CVE.

marek22k commented 3 months ago

I don't know that either (without further research), as it wasn't included in the tailscale dashboard.

marek22k commented 3 months ago

I would understand if Tailscale were to be removed from OpenWrt - but I would still find it a shame. (Well, you can also install it manually - if the router has enough memory).

I looked at the changelogs of Tailscale and only found one UPnP vulnerability (TS-2023-006) (or I missed the others).

Summit48 commented 3 months ago

I would understand if Tailscale were to be removed from OpenWrt - but I would still find it a shame. (Well, you can also install it manually - if the router has enough memory).

I currently manually install Tailscale on Ubiquiti EdgeRouters running EdgeOS. Can you provide information on how to manually install Tailscale on the Ubiquiti USG-3P running OpenWrt?

Thanks

marek22k commented 3 months ago

Maybe take a look at https://tailscale.com/kb/1053/install-static ?!

Summit48 commented 3 months ago

Thanks I already use Static Binaries in conjunction with the following installation procedures for EdgeOS.

https://github.com/jamesog/tailscale-edgeos or https://gist.github.com/lg/6f80593bd55ca9c9cf886da169a972c3

I was looking for OpenWrt equivalent installation procedures to use with the Static Binaries.

brada4 commented 3 months ago

installation procedures to use with the Static Binaries

You will not find those via bug tracker. Best attempt is to backport tailscale to older Go by reverting ivia a apatch ncompatible dependabot changes upstream.

raggi commented 2 months ago

The developers of the commercial product Tailscale could offer their assistance by releasing OpenWrt packages for stable versions or by providing support and solutions for OpenWrt. Otherwise, Tailscale may be removed from OpenWrt. Additionally, they could release a new version of Tailscale with the security fix that supports Golang 1.21. It is important to note that Golang 1.21 is still a supported version (https://go.dev/doc/devel/release - see "Release policy".

@BKPepe Tailscalar here. I'm curious what we could do here. Would you be willing to accept prebuilt binaries from us? We have static builds for Linux available.

mochaaP commented 2 months ago

@raggi I don't think we could accept prebuilt binaries in OpenWrt. If you would like to ship the static binaries for OpenWrt users, the best possible way is to create a opkg repository.

SuperSandro2000 commented 2 months ago

Best attempt is to backport tailscale to older Go by reverting ivia a apatch ncompatible dependabot changes upstream.

That is a fruitless activity. Golang is an absolute bitch about what go version dependencies use. If any dependency bumps their go directive to 1.23, you have some trouble getting it to build.

Right now we could downgrade it to golang 1.22, thats the version go mod tidy will write into it but then go build will complains that newer features are used and the build will fail anyway.

But doing that is pretty standard for many golang projects and something commonly done. Openwrt could just provide a 2nd go package that has the latest version on stable and the problem is solved.

PS: Debian basically has the same problem with golang and rust constantly and stopped shipping the normal Firefox version of that and now you need to fiddle with 3rd party repos or stay on the dated ESR.

brada4 commented 2 months ago

Thats only go.mod patch on top of release tarball then?

SuperSandro2000 commented 2 months ago

Well, if no library demands a newer version and no new language features are used, yes, but unfortunately tailscale is already using new features so you would need to rewrite those functions to use the old syntax.

brada4 commented 2 months ago

I dont need anything from tailscale, you need the new compiler you make it.

SuperSandro2000 commented 2 months ago

There is already #24992 and that just needs to be backported or copied to stable with a different PKGNAME.

Summit48 commented 2 months ago

@raggi I don't think we could accept prebuilt binaries in OpenWrt. If you would like to ship the static binaries for OpenWrt users, the best possible way is to create a opkg repository.

Why are prebuilt binaries unacceptable?

BKPepe commented 1 month ago

Would you be willing to accept prebuilt binaries from us? We have static builds for Linux available.

Since we are living in AI world, I asked Copilot for that, because it provides the answers which you are looking for. image

We will not need to discuss all those points, which are there and all of them are really rock solid. Well, I haven't seen that we accepted prebuild binaries for other packages, and we will not make any exceptions here, though. I am not sure how we should integrate your prebuild binaries to our repository. 🤷 I think, it is the same as for other GNU/Linux distributions; they don't have tail scale either. More details in the referenced issues/pull requests in my comment.

Providing stable releases (most likely as LTS) and the development versions makes sense for us, and it will help others, too.

SuperSandro2000 commented 1 month ago

I think, it is the same as for other GNU/Linux distributions; they don't have tail scale either.

I see Alpine, Gentoo, NixOS, OpenBSD, OpenSuse, VoidLinux to just name a few https://repology.org/project/tailscale/versions

marek22k commented 1 month ago

I think, it is the same as for other GNU/Linux distributions; they don't have tail scale either.

I see Alpine, Gentoo, NixOS, OpenBSD, OpenSuse, VoidLinux to just name a few https://repology.org/project/tailscale/versions

And Arch. If I'm not mistaken, the reason it wasn't included in Debian is that Debian doesn't provide fast enough updates for Tailscale (according to the Tailscale developers).

BKPepe commented 1 month ago

Thanks for link, @SuperSandro2000. Have you checked how many of those GNU/Linux distributions have the latest tailscale version? 😇 There is a good exmaple that you can check current OpenWrt versions and against others.

SuperSandro2000 commented 1 month ago

We really should not look at Debian when searching for innovative ideas and to find solutions. Debian has many problems with Go binaries since the beginning due to stupid decisions in their build system (splitting binaries into shared objects) and by using features upstream doesn't really supported. For example docker was severely outdated for ages because of that. Also Debian doesn't support normal versions of browsers on their stable release which are pretty essential in the time we are living.

Have you checked how many of those GNU/Linux distributions have the latest tailscale version?

You mean the 3 EOL versions of alpine and the 4 of nixos the site lists? Probably half of the supported distros have the latest version which is pretty great if I must admit.

There is a good exmaple that you can check current OpenWrt versions and against others.

that would be pretty easy to fix if there would be go123 or at least go 122 and then we would be rocking.

scotjam commented 1 month ago

Please don't drop the package if possible as it's incredibly useful for openwrt - running on routers is one of the things that tailscale is really great for

BKPepe commented 1 month ago

Running Tailscale on a router can be problematic if the Tailscale package itself becomes outdated and vulnerable. In this thread, we tried to explain what was happening and said that always having the latest tailscale versions is impossible. Tailscale's developers said loudly and clearly why the tailscale is not in many GNU/Linux distributions. It would help if you tried to convince them to be more involved in the community. I appreciate all the comments here that we should not remove it, but has anyone considered the option that we are an open-source project? We have many volunteers here who do it in their free time. Any help is appreciated, and we haven't seen anyone who would step in and improve the current situation.

From this discussion, I haven't seen any conclusion as to why this package should stay in. As it was said elsewhere. We can not compete with other GNU/Linux distributions, as said here: https://github.com/openwrt/packages/pull/25062#issuecomment-2385239836, and there are reasons why some packages have stable (most known as LTS) and the latest one. Guys, you should not forget that tailscale is built on top of WireGuard and WireGuard is included in the Linux kernel. There is no way how to fix the OP issue in the stable branches.

scotjam commented 1 month ago

Thank you for your reply BKPepe and for all of your hard work. I wish I had the capability to contribute but I don't even know what Golang is or how it relates to tailscale/openwrt. Is that the language Tailscale is written in? Or is it a library?

For what it's worth, my project worked thanks to all the documentation of openwrt/tailscale available in various places. My project was getting one subnet router running openwrt/tailscale at my place and an exit node router at my inlaws running openwrt/tailscale so the inlaws can connect to the router at my place and everything works as if they're at home (their streaming services etc). I came past this thread when I saw the security warning in the tailscale interface because I didn't want to leave an insecure router in their home.

I managed to get tailscale updated to 1.72.1-1 (OpenWrt) by compiling a custom openwrt image so it is no longer flagged as being insecure.

Just in case anyone really needs to update this and (like me at the start of this) doesn't know how to compile a firmware, these are the instructions I followed (must use debian, not ubuntu): https://pastebin.com/Wvksa9ih

I don't think I could have done this at all if tailscale was dropped from the openwrt repositories (maybe someone could but I don't think I would have figured it out).

I would love to help convince the tailscale developers to be more involved as well - do you know how I could go about doing that? forum.tailscale.com seems to have a note that it is read-only as of now.

"Guys, you should not forget that tailscale is built on top of WireGuard and WireGuard is included in the Linux kernel. There is no way how to fix the OP issue in the stable branches." - does that imply it can be fixed for users using something other than stable branch? I mean that feels like a good compromise if it is possible.

raggi commented 1 month ago

@raggi I don't think we could accept prebuilt binaries in OpenWrt. If you would like to ship the static binaries for OpenWrt users, the best possible way is to create a opkg repository.

I'd like to get a sense of scope for this - is there documentation on the structure of opkg repositories somewhere? I've been looking over the Wiki, but haven't found a document there, and the wiki seems to be having an extremely slow serving day. I'm looking for something like this: https://wiki.debian.org/DebianRepository/Format

It would help if you tried to convince them to be more involved in the community.

One potential solution I can see here is to build a suitable Go toolchain in the Tailscale package build. This is essentially the bootstrapping path taken in a lot of builds. I suspect that might somehow run afoul of a single-version policy, but if I provided a tarball with a single build entrypoint containing all of the relevant code and requiring only a bootstrap Go toolchain would that be acceptable and useful?

brada4 commented 1 month ago

If you peovide exact cve + patch/commit the security issue can be fixed in old versions.

SuperSandro2000 commented 1 month ago

said that always having the latest tailscale versions is impossible.

There is no way how to fix the OP issue in the stable branches.

No, it isn't impossible. There are just rules in place which forbid the simplest and most time effective option to just backport golang 1.23 as golang-1-23 and build tailscale with it. It has no dependencies on any shared object other than the linker and libgcc_s which probably can also be removed when compiling with CGO_ENABLED=0.

With that in place, tailscale could then be easily be backported with very low effort.

Currently this is not allowed in the rules and I see this as a very big limitation holding us back in the possible options we have here.

I haven't seen any conclusion as to why this package should stay in.

I don't get that question. Because people use tailscale and want to continue to use it?

mochaaP commented 1 month ago

will backport the fixes soon. @raggi is it possible for tailscale to mark OpenWrt versions as "probably vulnerable" since this is not usually the case on OpenWrt systems? users are confused about this.