Would you consider adding a nix flake installation?

SimplyKnownAsG commented 3 days ago

(This is my own interpretation:) Nix is a declarative tool for creating reproducible and reliable builds. Nix can function as a package manager on Linux operating systems (including Amazon Linux) in addition to OSX (https://nixos.org/). A user can install a nix flake by pointing to the correctly structured repository (e.g. contains a flake.nix and flake.lock). By maintaining the flake.nix and flake.lock file in the existing repository, nix users can install the ion-cli with little maintenance from the ion-cli team.

I have a fork here, where I've added the two files flake.nix and flake.lock. The flake.nix should be mostly stable in this state as it doesn't actually run tests, because it fails the roundtrip tests that require gradle. I believe this is consistent with the Dockerfile that appears to only install but not run tests. The flake.nix also uses nixos-unstable, which was required to get rustc 1.80. The flake.lock should be updated periodically updated.

This uses nixpkgs, which have releases every six months, 24-05 being the latest (24-11 will be released this month). For a more stable release, flake.nix should switch nixos-unstable to 24-05 or some other stable tag .

Issues

This does not run tests. I have another with-checks branch that has doCheck = true, but it is not working because gradle appears to override the buildPhase and checkPhase.

Nix configuration

You'll need nix installed. You'll need it configured with flakes.

Nix testing

There's a couple different things you can do.

nix develop -i will open a clean bash shell without any inherited environment. You can run cargo build, for example, and it will build. cargo test will fail until we can figure out how to get gradle without iterfering.

nix build will actually build it.

Installation

The installation process looks like this

# user's flake.nix
{
   ...
   inputs = {
       ...
       ion-cli = {
           url = "github:amazon-ion/ion-cli";
       };
   };

   ...
   pkgs = [
    ion-cli
   ];
}

popematt commented 3 days ago

I've been looking at the Nix documentation for Flakes to try to understand this better, and there are a few things that look concerning.

At the same time, flakes have fundamental architectural issues and a number of problems with the implementation, and there is no coordinated effort to resolve them systematically.

As an outsider to Nix, I don't know how serious this is, but when a project's own documentation is calling it out with language this strong, that gives pause for caution.

In principle, the same level of reproducibility can be achieved with or without flakes. In particular, the process of adding software to Nixpkgs or maintaining NixOS modules and configurations is not affected by flakes at all. There is also no evidence that flakes could help solving the scalability challenges of either.

This leaves me wondering why we should use flakes at all if reproducible builds are possible even without them.

The Nix documentation team focuses on improving documentation and learning materials for stable features and common principles. When using flakes, you will have to rely more heavily on user-to-user support, third-party documentation, and the source code.

It looks like flakes are an experimental feature with admittedly incomplete documentation and an unclear value proposition. The lack of documentation for an experimental feature is the most concerning because we would not be equipped to maintain the flakes files and anything we do learn about Flakes could become irrelevant as the Flakes API evolves. The consequence is that we are (a) going to be accidentally breaking the Flake files as we make changes to ion-cli, or (b) going to have to delay improvements to ion-cli while we figure out the necessary changes to the Flake files. So, unless I have misunderstood something, we don't really want to have Flakes in the ion-cli repository.

Does it work with the Nix ecosystem to host the Flakes files in a different repo?

If you want to create a repo (e.g. SimplyKnownAsG/nix-flakes-for-ion-cli) that's a thin wrapper with the relevant Nix configuration, we would be happy to link to that on the Ion website and in the Ion CLI README as a community-supported installation method.

SimplyKnownAsG commented 2 days ago

Hahaha. Well, I guess you've got a valid point there. But, it's still better than dealing with the airlines.

Yes, we can definitely host it in a separate repo, or just bundle it into nixpkgs.

Given that the intersection between ion-cli users and Nix users may be ~4 people, I might hold off on committing much effort as well. Though, I'll probably keep what I have working.

amazon-ion / ion-cli