search

NHAS / wag

Simple Wireguard 2FA
BSD 3-Clause "New" or "Revised" License
499 stars 27 forks source link

Use an alpine 3.17 LXC container for small footprint #58

Closed Wyk72 closed 1 year ago

Wyk72 commented 1 year ago

Just in case someone needs it, I'm running WAG into an LXC container that is just 38Mb.

Works beautifully.

That's how I did it:

GLIBC_VERSION="2.34-r0"

apk add --update curl && \
  curl -Lo /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub && \
  curl -Lo glibc.apk "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-${GLIBC_VERSION}.apk" && \
  curl -Lo glibc-bin.apk "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-bin-${GLIBC_VERSION}.apk" && \
  mv /etc/nsswitch.conf /etc/nsswitch.conf.bak && \
  apk add --force-overwrite glibc-bin.apk glibc.apk && \
  /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc-compat/lib && \
  mv /etc/nsswitch.conf.bak /etc/nsswitch.conf && \
  apk add iptables linux-pam && \
  sysctl -w net.ipv4.ip_forward=1 && \
  wget https://github.com/NHAS/wag/releases/download/v7.3.1/wag && \
  chmod +x wag && \
  apk del curl && \
  rm -rf glibc.apk glibc-bin.apk /var/cache/apk/*

You'll end with a wag binary that works and does all his beautiful stuff.

Of course you have to configure the networking part of the container as you see fit

Enjoy.

NHAS commented 1 year ago

Howdy,

I wouldn't really think this is an issue per say. Looks more like a pull request if you wanted to submit it.

Just curious, why doesn't this work on 3.18?

(additional side note the image as it currently stands is only 68mb)

NHAS commented 1 year ago

Ah actually I see you're pulling in random sources from github.

I'd prefer not to do that

Wyk72 commented 1 year ago

I am not "randomly pulling" anything.

It's basically the same script that builds the docker version af Alpine w glibc, that has 5+ million pulls on docker (and nobody bats an eye), but in basic LXC fashion, since Docker is not really my cup of tea.

https://hub.docker.com/r/frolvlad/alpine-glibc/

I've noticed also that only Alpine 3.17 and glibc 2.34-r0 work as expected: Alpine 3.18 (or edge) make the wag binary segfault:

[  505.508798] wag[3546]: segfault at 3590 ip 0000000000003590 sp 00007ffdca3f7da8 error 14 in wag[400000+8000] likely on CPU 0 (core 0, socket 0)
[  505.508835] Code: Unable to access opcode bytes at 0x3566.

Difficult to say what is gong wrong: I suspect this has something to do with an Alpine 3.18 change in linker options to shrink binaries a little more. From changelog 3.17 to 3.18 I read:

DT_RELR

on x86, x86_64, and ppc64le, -Wl,-z,pack-relative-relocs is now added to the base LDFLAGS. this reduces elf (executable/shared-library) size by 10% on average.

however, these binaries are now not portable to other musl-based systems that have a musl older than 1.2.4, because older versions are incapable of loading these binaries.

Maybe I'll hunt it down w a debugger when I feel inspired: suffice to say 3.17 (and lower, 3.16 3.14) all work as expected.

I was wondering if it's possible to build the WAG binary as static to avoid the never-ending chase for packages, binaries & secret recipes to get things working.

NHAS commented 1 year ago

I am not "randomly pulling" anything.

It's basically the same script that builds the docker version af Alpine w glibc, that has 5+ million pulls on docker (and nobody bats an eye), but in basic LXC fashion, since Docker is not really my cup of tea.

Hi there, sorry I should have been clearer in my language. This setup script is including arbitrary resources from a github repository that is outside of my control and not part of a standard image release (such as alpine pure). As this product is for use in a security consultancy (see the sponsorship from https://www.aurainfosec.com), I am reluctant to use anything that could be subject to a supply chain attack.

It is worth noting that when I bump the golang libraries I use, I check the sources (if they are not in the stdlib itself).

I can understand how this came off, but unfortunately I need to do extreme due diligence when making changes or accepting changes.

NHAS commented 1 year ago

I would love to build wag purely static, unfortunately supporting sqlite3 in its current form comes with the C dependency which means we have issues. I did try and use the pure golang implementation of sqlite3, but it had both weird performance problems and had contention issues when writing to the db on client ip address change.

NHAS commented 1 year ago

In my ongoing war to hopefully make wag HA at some point which may require moving to postgres we may see the C dependency be dropped at some point. But not any time soon due to workload.

Wyk72 commented 1 year ago

I perfectly understand your point of view: my post was just an idea for a "home usage" of your program, (in my case was to embed it into a router with mere 512Mb of Ram and 256MB flash, so I had to shrink it further, and avoid docker).

Anyway I think I spotted the "culprit" of the segfault: to make it work on Alpine 3.18+ a glibc 2.36 package is needed to support this ELF binary format change (duh).

From glibc changelog:

"Version 2.36

Major new features:

* Support for DT_RELR relative relocation format has been added to
  glibc.  This is a new ELF dynamic tag that improves the size of
  relative relocations in shared object files and position independent
  executables (PIE).  DT_RELR generation requires linker support for
  -z pack-relative-relocs option, which is supported for some targets
  in recent binutils versions.  Lazy binding doesn't apply to DT_RELR."

I'll try it today, if I find a 2.36 glibc package for Alpine, just for fun.

NHAS commented 1 year ago

Sounds like a good time! Im interested that the change they've implemented there broke the program/glibc

Wyk72 commented 1 year ago

It was exactly that.

After struggling a few hours I've built a 2.37 glibc binary (used https://github.com/sgerrand/docker-glibc-builder, a docker recipe that generates a 2.37 glibc binay archive) and "upgraded" it into the alpine container by brutally overwriting binaries with tar to the container rootfs. Ugly as f, but worked, so I can confirm that

The binary WORKS under Alpine 3.18 with glibc 2.37

This glib/musl dichotomy is such a painful affair, binaries in general are extremely painful stuff.

That's a big part of docker success, imo.