Building Crates in Nix

[atalii]

Nix has some somewhat aligned goals to Alire, but goes about them at the level of a package manager rather than language-level tooling. Nix can be used to wrap some build tools (traditionally autotools, but there's good integration with cmake, meson, cargo, go, &c) to enable and provide reproducible packaging.

I would like to be able to package Alire crates in Nix. To do so, I'm testing with the following derivation:

pkgs.stdenv.mkDerivation {
  pname = "test-crate";
  version = "0.1.0";
  src = ./.;

  nativeBuildInputs = with pkgs; [ gprbuild gnat alire ];

  buildPhase = ''
     alr build
  '';

  installPhase = ''
    cp -r bin $out/bin
  '';
}

If you run this, you run into these two issues first:

1. $HOME is set to /homeless-shelter. For Alire not to complain, we have to unset HOME prior to alr build. Alternatively, Alire could automatically fallback to a /tmp directory if $HOME doesn't exist and can't be created --- would that be a good thing for it to do? I'd be happy to send a PR.
2. The build environment doesn't contain git. Of course, Alire quits execution as soon as it finds that out. We could just install git in nativeBuildInputs so that it's available, but then we run into a more fundamental issue: Nix doesn't allow derivations to access the internet*, and the first thing Alire tries to do is fetch the index.

_{*Fixed-output-derivations allow for internet access in exchange for a hash of the expected output to ensure reproducibility. Nix wrappers of, e.g., Go and Cargo each use this to build a 'vendored' version of their packages, and then perform an offline build.}

The broader issue, however, appears to be that there is no existing mechanism for offline and sandboxed builds. Thus, two things primarily are required to support Nix:

1. A way to vendor crate dependencies alongside the code.
2. A way to build offline, assuming deps are vendored.

There's also a secret third extra difficulty in that Nix can't run binaries downloaded from the internet due to dynamic linking issues. This is obviously very much at odds with Alire's toolchain mechanism. I would think that part of an offline build involves using the native or local toolchain, but that doesn't strike me as a complete solution.

---

Is this impression correct? Am I understanding the problem space well; i.e., is this all required to build Alire crates with Nix? Do you think I'm missing anything or making it more complicated than it needs to be?

If this or something similar looks reasonable, I think I should be able to devote some time to implementing the changes required to get Alire working in this setting. I would appreciate some help, however, in strategizing a way to do this all incrementally and cleanly. Thanks for any input you could provide!

[mosteo]

For Alire not to complain, we have to unset HOME prior to alr build.

I presume that /homeless-shelter is a kind of /dev/null?

1. Alternatively, Alire could automatically fallback to a /tmp directory if $HOME doesn't exist and can't be created

I don't see a problem here, this should be a minor patch.

the first thing Alire tries to do is fetch the index.

This can be disabled via alr settings for automatic refresh, and as long as you configure an index as the first thing, it won't try to add another one. Our testsuite, for example, is not using any online index (and actually no tests should go online except a few that explicitly are marked as such).

there is no existing mechanism for offline and sandboxed builds.

We should explore this (mis?)conception in more detail, because I don't think there's anything preventing this on principle. Actually someone from AdaCore recently submitted a test to verify that builds can be performed in air-gapped systems, see https://github.com/alire-project/alire/pull/1760.

A way to vendor crate dependencies alongside the code.

With vendored dependencies, I'm asuming you mean local copies of dependencies? Alire has specific ways to access such dependencies for "pulling" (using a local folder or tarball). Another possibility would be to "pin" to the sources prior to building. This would require a custom index or some scripting.

I would think that part of an offline build involves using the native or local toolchain, but that doesn't strike me as a complete solution.

If Nix can provide gnat and gprbuild, then these can also be used by Alire, bypassing any remote toolchain. I'm not sure I follow the implications of this point.

On the general intent, I'm happy to help with this, and I think the rough building blocks are already there. It may require some scripting, but maybe not even big changes to Alire itself. Of course, if some new feature is needed that would save you lots of time, that's the first thing that we should identify.

I guess the next steps would be to have a minimal proof-of-concept? And I can try to help you with any blockers?

[atalii]

Thanks for the quick reply - I very much appreciate you correcting most of my misconceptions. I'll send a PR to fallback if $HOME doesn't have adequate perms/can't be created, and then I'll try to put together a proof of concept based on that airgapped test.

We should explore this (mis?)conception in more detail, because I don't think there's anything preventing this on principle.

Honestly, I think I just didn't know about the mechanisms that do exist. I'll look at what that test does and see if I can't replicate it in Nix - thanks for proving me wrong there.

If Nix can provide gnat and gprbuild, then these can also be used by Alire, bypassing any remote toolchain. I'm not sure I follow the implications of this point.

Cross-compilation and the like come to mind. There are cases where the toolchain specified by Alire isn't the toolchain that Nix provides, and I'd want to be able to address those. That said, it's not a crucial point yet.

Again, thanks for the detailed response - I'll be back with a bit more code :)

[mosteo]

Cross-compilation and the like come to mind. There are cases where the toolchain specified by Alire isn't the toolchain that Nix provides, and I'd want to be able to address those.

OK, now I understand.

Good luck with the PoC!