Support Nix depexts with `opam env`

[RyanGibb]

Nix doesn't install packages in the traditional sense -- instead it puts them in a store and makes them available through environment variables. I.e., nix-shell -p gcc will drop us into a shell with gcc in the $PATH.

We can set appropriate environment variables with Opam to make system dependencies (depexts) available with Nix. Similar to how nix-shell works under the hood, we create a Nix derivation such as

{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
  packages = [ gmp ];
  inputs = with buildPackages; packages ++ [ pkg-config ];
in
stdenv.mkDerivation {
  name = "opam-nix-env";
  nativeBuildInputs = inputs;

  phases = [ "buildPhase" ];

  buildPhase = ''
vars=("NIX_CC" "NIX_CC_FLAGS" "NIX_CFLAGS_COMPILE" "NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu" "NIX_LDFLAGS" "PKG_CONFIG_PATH")
for var in "''${vars[@]}"; do
  escaped="$(echo "''${!var}" | sed -e 's/^$/@/' -e 's/ /\\ /g')"
  echo "$var    =       $escaped        Nix" >> $out
done
echo "PATH      =       $PATH   Nix"
  '';

  preferLocalBuild = true;
}

Which we can build to output a file with environment variables that make depexts available, in Opam's environment variable format. This file is a Nix store root, so it's dependencies won't be garbage collected by Nix until the file is removed.

This approach came from conversations with @dra27 and is distinct from previous approaches in that it supports providing development environment, which imperative installations with Nix don't https://github.com/ocaml/opam/pull/5332#issuecomment-1710409758; and doesn't require the user to manage the environment outside of Opam, which would lead to a different workflow https://github.com/ocaml/opam/pull/5942.

Initial experiments were done using a package to set such environment variables https://github.com/RyanGibb/nix.opam. However, in order to work with generic depexts (as opposed to just conf packages), to avoid cyclic dependencies (if conf packages depend on nix.opam but nix.opam depends on conf packages), and due to opam package sandboxing (nix derivation fetching would require opam sandboxing to be disabled or the nix store to be 'primed'), it's better implemented as a depext mechanism.

This has been tested and successfully creates an environment to provide packages conf-gmp and conf-libseccomp, as well as depexts directly.

[RyanGibb]

Rebased on master

[RyanGibb]

Looks like I'm running into https://github.com/NixOS/nix/issues/10587 with the Docker image

[RyanGibb]

Rebased on master -- and could do with another depext CI run if it needs re-approval!

[RyanGibb]

Okay! The CI is failing only due to https://github.com/ocaml/opam-repository/pull/26261 (I've got it working locally with my branch of opam-repository)