Add Nix builds for the full registry

[thomashoneyman]

Extends #618 by additionally providing:

• A Nix build for every package that can be run in the registry (the github importer, the registry server, the various scripts like the legacy importer, the package transferrer, etc.)
• A NixOS module that defines a full server. The registry-server service is packaged as a systemd unit. We can deploy this module remotely, to our Digital Ocean NixOS droplet, or we can deploy it to a local VM with nix run. From there you can open localhost:8080 in your browser, or send requests to localhost:8080/api/v1/<path> with something like curl.
• Repository tests as part of the flake checks, so things like code formatting and dhall verification are run the same way locally as they are in CI — and much faster, since Garnix handles both of those checks in ~5 seconds and in parallel.
• A full integration test with a client making requests to the registry server, which is also run in the flake checks. We can expand on this to have a whole suite of integration tests, which I would very much like.

What's the point of all this? In short:

• to be able to run and test the registry server exactly the way it will run on the deployed server, without actually deploying it
• to be able to automatically deploy the server when CI is successful, knowing the integration tests are running on the same system configuration as the deployed target
• to have significantly faster CI with garnix

Note that all packages in the registry can still be run locally. You don't have to use a VM. Things still work fine in the GitHub Actions environment.

You can run any of the existing scripts with their module name swapped to lower-cased, hyphenated attribute names. For example, Registry.Scripts.LegacyImporter:

nix run .#legacy-importer -- dry-run

Or the server (not in a virtual machine), assuming you have your env vars set up properly:

nix run .#server

You can deploy the server to a VM and run it on localhost:8080, and then load that in your browser or curl to the api endpoint:

nix run

You can still get into your Nix shell as usual:

nix develop

[thomashoneyman]

The latest commits add a full integration test using the local VM. That local module definition should also be sufficient to implement deployments to our Digital Ocean server, so I'll take care of that next. At that point all that's left to do is polish this up!

[thomashoneyman]

The latest commit supports deployments to the registry NixOS server on Digital Ocean — you can deploy with colmena apply. The first deployment takes forever because we build on the target machine, and I got an underpowered machine for testing, but subsequent deploys take 2-3 minutes or so if you change the PureScript source code.

Available right now here: http://161.35.111.85

For example:

$ curl 161.35.111.85/api/v1/jobs/0
TODO

Next I'll set up certificates and auto-deploys on commits to master when all the garnix checks complete successfully. At that point we're all set up for continuous deployments.

[thomashoneyman]

I've verified that we can deploy with Let's Encrypt and force secure connections, though with a bare IP address the certs are invalid. I've commented them out, but once we have an e.g. https://registry.purescript.org domain then we can enable it again.

[thomashoneyman]

Note: deployments are working from non-x86_64-linux systems because we're running the build on the server, not locally. However, you won't be able to run the integration test on a darwin system unless you're running a remote builder as per https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder

I'm going to assume no one is doing that so I'm going to disable the integration test for darwin systems. It will still run in CI, so you can still open a PR to run it. It will also be runnable on any Linux system.

[f-f]

This looks good - one last thing that we should take care of before merging is some kind of developer documentation, i.e. how to use all of this Nix machinery for folks that are not intimately familiar with Nix and/or with this specific setup.

You added some details in the PR description, but let's have a CONTRIBUTING.md file with instructions on how to run tests, deploy, etc.

[thomashoneyman]

Developer documentation is on the way in the next PR.