Literally a complete rewrite, devouring whole thing with Nix and wg-bond

cab404 commented 2 years ago

to: cc: @subspacecommunity/subspace-maintainers related to: resolves:

Background

Reason for the change: It wasn't working as we wanted it to.

Changes

Replace logic module with wg-bond, to support arbitrary network masks
Replace all the docker with lack of docker

Testing

We tested it extensively.

agonbar commented 2 years ago

Hmmm I'd like to ask the rest of the @subspacecommunity/subspace-maintainers about this, as far as I know, most people use this service with docker.

I'd love to add support for Nix, but not at the expense of losing docker.

cab404 commented 2 years ago

We've PRd this half as joke, and half as a discoverability tool — and also we can build Docker images (requires some more work though) from that thing as well)

gchamon commented 2 years ago

heard some intresting things about nix, but its nature for system and deployment config is highly experimental AFAIK and the learning curve seems steep. What are the gains of potentally replacing the current set of automation toolings with nix?

notgne2 commented 2 years ago

NixOS isn't experimental and works just fine in production, but alas you almost definitely do not want it to only work as a NixOS module unless you plan to only use NixOS (and you should of course but we haven't quite taken over the world yet). But as for the packaging with Nix, it is fully reproducible by design, both in the sense of keeping each build pure, but also being able to locally replicate everything, it also means if you don't want Docker, either out of convenience, necessity, or principle, you don't need it, because Nix doesn't suffer from the problems that make Docker useful to begin with for many use cases.

This PR existing is as @cab404 said mostly a joke but if you're actually interested in upstreaming Nix/NixOS support it wouldn't be at the expense of Docker, we just removed it in our fork because we don't intend on ever using it, and it would have needed to be updated to support some changes we made.

We probably would have made a more "compatible" PR for Nix support, if it weren't for us trying to get it to work for us in a hurry (and thus breaking Docker), and the fact that we also gutted and bastardized all of the actual Wireguard code in here in favor of internally using wg-bond to make it work properly for our use cases too, we doubt this would be upstreamed.

If there's any actual interest in Nix support, we can at least upstream this part so so our diff doesn't have to be as big. We could also optionally alter the Docker stuff to just use Nix internally, which would provide some of the benefits of Nix, plus de-duplicate the packaging.

I don't remember the exact issues we had with the existing Wireguard logic, but I remember it being slightly broken, didn't work with some subnets, and didn't work with how we run/orchestrate Wireguard (I think @cab404 remembers more details about this), so we just gutted the whole thing. I think it's unlikely you would be interested in this part, though it has some benefits, so I guess let us know if you are.