fstar: fstar.lib not available anymore since NixOS 24.05 upgrade

someplaceguy commented 4 months ago

Describe the bug

I have a minimal default.nix created from the example in the NixOS 24.05 manual:

let
  pkgs = import <nixpkgs> {};
  # choose the ocaml version you want to use
  ocamlPackages = pkgs.ocamlPackages;
in
pkgs.mkShell {
  nativeBuildInputs = with ocamlPackages; [ ocaml findlib pkgs.fstar ];
}

When using NixOS 23.11, this works fine for building an F* program extracted to OCaml, because ocamlfind list can find the necessary dependencies (in particular, including fstar.lib):

$ nix-shell ./default.nix

[nix-shell:~/tmp/ocaml]$ ocamlfind list
base                (version: v0.16.2)
base.base_internalhash_types (version: v0.16.2)
base.caml           (version: v0.16.2)
base.md5            (version: v0.16.2)
base.shadow_stdlib  (version: v0.16.2)
batteries           (version: n/a)
batteries.top       (version: n/a)
batteries.unthreaded (version: n/a)
bigarray            (version: [distributed with Ocaml])
bytes               (version: [distributed with OCaml 4.02 or above])
camlp-streams       (version: n/a)
compiler-libs       (version: [distributed with Ocaml])
compiler-libs.bytecomp (version: [distributed with Ocaml])
compiler-libs.common (version: [distributed with Ocaml])
compiler-libs.native-toplevel (version: [distributed with Ocaml])
compiler-libs.optcomp (version: [distributed with Ocaml])
compiler-libs.toplevel (version: [distributed with Ocaml])
dynlink             (version: [distributed with Ocaml])
findlib             (version: 1.9.6)
findlib.dynload     (version: 1.9.6)
findlib.internal    (version: 1.9.6)
findlib.top         (version: 1.9.6)
fstar               (version: n/a)
fstar.lib           (version: n/a)
gen                 (version: 1.1)
menhirLib           (version: 20230608)
num                 (version: 1.0)
num-top             (version: 1.1)
num.core            (version: 1.0)
ocaml-compiler-libs (version: n/a)
ocaml-compiler-libs.bytecomp (version: n/a)
ocaml-compiler-libs.common (version: n/a)
ocaml-compiler-libs.optcomp (version: n/a)
ocaml-compiler-libs.shadow (version: n/a)
ocaml-compiler-libs.toplevel (version: n/a)
ocamldoc            (version: [distributed with Ocaml])
pprint              (version: n/a)
ppx_derivers        (version: n/a)
ppx_deriving        (version: n/a)
ppx_deriving.api    (version: 5.2.1)
ppx_deriving.create (version: 5.2.1)
ppx_deriving.enum   (version: 5.2.1)
ppx_deriving.eq     (version: 5.2.1)
ppx_deriving.fold   (version: 5.2.1)
ppx_deriving.iter   (version: 5.2.1)
ppx_deriving.make   (version: 5.2.1)
ppx_deriving.map    (version: 5.2.1)
ppx_deriving.ord    (version: 5.2.1)
ppx_deriving.runtime (version: 5.2.1)
ppx_deriving.show   (version: 5.2.1)
ppx_deriving.std    (version: 5.2.1)
ppx_deriving_yojson (version: 3.7.0)
ppx_deriving_yojson.runtime (version: 3.7.0)
ppxlib              (version: 0.30.0)
ppxlib.ast          (version: 0.30.0)
ppxlib.astlib       (version: 0.30.0)
ppxlib.metaquot     (version: 0.30.0)
ppxlib.metaquot_lifters (version: 0.30.0)
ppxlib.print_diff   (version: 0.30.0)
ppxlib.runner       (version: 0.30.0)
ppxlib.runner_as_ppx (version: 0.30.0)
ppxlib.stdppx       (version: 0.30.0)
ppxlib.traverse     (version: 0.30.0)
ppxlib.traverse_builtins (version: 0.30.0)
process             (version: 0.2.1)
result              (version: 1.5)
sedlex              (version: 3.2)
sedlex.ppx          (version: 3.2)
sedlex.utils        (version: 3.2)
seq                 (version: [distributed with OCaml 4.07 or above])
sexplib0            (version: v0.16.0)
stdint              (version: 0.7.2)
stdio               (version: v0.16.0)
stdlib              (version: [distributed with Ocaml])
stdlib-shims        (version: 0.3.0)
str                 (version: [distributed with Ocaml])
threads             (version: [distributed with Ocaml])
threads.none        (version: [internal])
threads.posix       (version: [internal])
unix                (version: [distributed with Ocaml])
yojson              (version: 2.1.1)
zarith              (version: 1.13)
zarith.top          (version: 1.13)

However, since upgrading to NixOS 24.05, ocamlfind list cannot find fstar.lib anymore (nor its dependencies):

$ nix-shell ./default.nix

[nix-shell:~/tmp/ocaml]$ ocamlfind list
bytes               (version: [distributed with OCaml 4.02 or above])
compiler-libs       (version: 5.1.1)
compiler-libs.bytecomp (version: 5.1.1)
compiler-libs.common (version: 5.1.1)
compiler-libs.native-toplevel (version: 5.1.1)
compiler-libs.optcomp (version: 5.1.1)
compiler-libs.toplevel (version: 5.1.1)
dynlink             (version: 5.1.1)
findlib             (version: 1.9.6)
findlib.dynload     (version: 1.9.6)
findlib.internal    (version: 1.9.6)
findlib.top         (version: 1.9.6)
ocamldoc            (version: 5.1.1)
runtime_events      (version: 5.1.1)
stdlib              (version: 5.1.1)
str                 (version: 5.1.1)
threads             (version: 5.1.1)
threads.posix       (version: [internal])
unix                (version: 5.1.1)

Any ideas what could be wrong?

Steps To Reproduce

Add fstar to the list of build inputs
Open a nix-shell
Run ocamlfind list and realize fstar.lib is not in the available list of libraries

Expected behavior

I expected fstar.lib to be listed in the ocamlfind list output.

Notify maintainers

cc @gebner @pnmadelaine

Add a :+1: reaction to issues you find important.

someplaceguy commented 4 months ago

To add to the mystery, if I add ocamlgraph to the list of dependencies, then ocamlfind list can find it without problems, so this issue seems to be specific to fstar.

But what's more baffling to me is that if I revert the latest fstar update, then the fstar derivation seems identical to the one I was using in NixOS 23.11 (which worked fine), and yet this issue still persists...

someplaceguy commented 4 months ago

I figured it out. The problem was this line:

  ocamlPackages = pkgs.ocamlPackages;

Apparently the default ocamlPackages version in NixOS 23.11 matched the ocaml version used for building fstar (4.14). However, in NixOS 24.05 the default ocaml version is now 5.1, which means that ocamlfind was looking for libraries built for ocaml 5.1 and therefore could not find the fstar.lib library built with ocaml 4.14.

So the fix was to change that line into this one instead:

  ocamlPackages = pkgs.ocaml-ng.ocamlPackages_4_14;

NixOS / nixpkgs