nix-shell failure

[paulyoung]

I'm encountering the following when trying to enter a nix-shell.

macOS Monterey, Version 12.0.1 (21A559)

$ nix-shell                                                                         ~/projects/dfinity/motoko/main
these 28 derivations will be built:
...
+ ocamlfind ocamlopt -c -g -bin-annot -safe-string -I src -I test -o src/uunf_data.cmx src/uunf_data.ml
ocamlopt.opt got signal and exited
Command exited with code 2.
pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
     '-build-dir' '_build' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md'
     'README.md' 'src/uunf.a' 'src/uunf.cmxs' 'src/uunf.cmxa' 'src/uunf.cma'
     'src/uunf.cmx' 'src/uunf.cmi' 'src/uunf.mli' 'src/uunf_data.cmx'
     'src/uunf_tmapbyte.cmx' 'src/uunf_tmapbool.cmx' 'src/uunf_tmap.cmx'
     'src/uunf_fmt.cmx' 'src/uunf_string.a' 'src/uunf_string.cmxs'
     'src/uunf_string.cmxa' 'src/uunf_string.cma' 'src/uunf_string.cmx'
     'src/uunf_string.cmi' 'src/uunf_string.mli' 'test/unftrip.native'
     'doc/index.mld' 'DEVEL.md' 'test/examples.ml']: exited with 10
error: builder for '/nix/store/bz3mcx2d3k6vakgy6y80vlmrn696xbyp-ocaml4.12.0-uunf-14.0.0.drv' failed with exit code 1;
       last 10 log lines:
       > Command exited with code 2.
       > pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
       >      '-build-dir' '_build' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md'
       >      'README.md' 'src/uunf.a' 'src/uunf.cmxs' 'src/uunf.cmxa' 'src/uunf.cma'
       >      'src/uunf.cmx' 'src/uunf.cmi' 'src/uunf.mli' 'src/uunf_data.cmx'
       >      'src/uunf_tmapbyte.cmx' 'src/uunf_tmapbool.cmx' 'src/uunf_tmap.cmx'
       >      'src/uunf_fmt.cmx' 'src/uunf_string.a' 'src/uunf_string.cmxs'
       >      'src/uunf_string.cmxa' 'src/uunf_string.cma' 'src/uunf_string.cmx'
       >      'src/uunf_string.cmi' 'src/uunf_string.mli' 'test/unftrip.native'
       >      'doc/index.mld' 'DEVEL.md' 'test/examples.ml']: exited with 10
       For full logs, run 'nix log /nix/store/bz3mcx2d3k6vakgy6y80vlmrn696xbyp-ocaml4.12.0-uunf-14.0.0.drv'.
error: 1 dependencies of derivation '/nix/store/634a6ri5zdyl58xk8ryg7x20dzv3z4i8-ocaml4.12.0-uucp-14.0.0.drv' failed to build
error: build of '/nix/store/1l8f8sccnxrpslzxpr8vbcfzx7y1cjin-ocaml4.12.0-ppx_tools_versioned-5.4.0.drv', '/nix/store/2kfn5zp27n79kil34yp0q2isvigq1dvr-ocaml4.12.0-ocamlformat-0.19.0.drv', '/nix/store/48gwh4k52881xi07v2w7fxzx5pddiwlq-drun.drv', '/nix/store/58xqsnaf68v35akbfvc707h9f1mf5k84-ocaml4.12.0-merlin-4.4-412.drv', '/nix/store/634a6ri5zdyl58xk8ryg7x20dzv3z4i8-ocaml4.12.0-uucp-14.0.0.drv', '/nix/store/6shmcgg9n1n0l7c0sdd6bb170d2966w0-wasmtime-0.21.0.drv', '/nix/store/ccr88y8wkar8qfpij4i4vkq1lha7slcp-ic-ref-run.drv', '/nix/store/f5mrcazzd4ax7ih63a1gh90zfcvhwqxp-niv-0.2.19.drv', '/nix/store/h8srnrkcni5xjkyamh5w6a9f2ysh834p-qc-motoko-1.drv', '/nix/store/jv1fc3z0rki4z1d6as3pipqcwfyfgfw0-rust-1.59.0-nightly-2021-12-01-48a5999fc.drv', '/nix/store/jzfh2szjc3pbracm3ryzwd21r8zhbdk9-lsp-int-0.drv', '/nix/store/qqack1dx9sl7a55ql9cxxbmic3fy0mjx-ocaml4.12.0-ocaml-migrate-parsetree-1.8.0.drv', '/nix/store/wfjvsz5viwbmxnx2b9frr4iyrdqhcka0-wasm-profiler-instrument.drv' failed

[paulyoung]

Perhaps https://github.com/NixOS/nixpkgs/pull/153686 will help. I'll give it a try.

[paulyoung]

That worked but then I ran into this:

  = note: Undefined symbols for architecture arm64:
            "_readdir$INODE64", referenced from:
                yanix::sys::bsd::dir::iter_impl::h2f2c32a20989b80a in libyanix-699c0fa6ed60634e.rlib(yanix-699c0fa6ed60634e.yanix.993e2a33-cgu.12.rcgu.o)
            "_fstat$INODE64", referenced from:
                yanix::file::fstat::h2c7cb40e007c0839 in libyanix-699c0fa6ed60634e.rlib(yanix-699c0fa6ed60634e.yanix.993e2a33-cgu.0.rcgu.o)
            "_fstatat$INODE64", referenced from:
                yanix::file::fstatat::had9f7a12369ea7f3 in libwasi_common-9bd597a51adbffdc.rlib(wasi_common-9bd597a51adbffdc.wasi_common.a51b261f-cgu.6.rcgu.o)
          ld: symbol(s) not found for architecture arm64
          clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

   Vendoring socket2 v0.4.0 (/private/tmp/nix-build-drun-vendor.tar.gz.drv-0/rs/cargo-home.iH4/registry/src/github.com-1ecc6299db9ec823/socket2-0.4.0) to drun-vendor.tar.gz/socket2
   Vendoring spin v0.5.2 (/private/tmp/nix-build-drun-vendor.tar.gz.drv-0/rs/cargo-home.iH4/registry/src/github.com-1ecc6299db9ec823/spin-0.5.2) to drun-vendor.tar.gz/spin
Lookup Uucp.Func.is_math for each uchar (10x)
   Vendoring spki v0.4.1 (/private/tmp/nix-build-drun-vendor.tar.gz.drv-0/rs/cargo-home.iH4/registry/src/github.com-1ecc6299db9ec823/spki-0.4.1) to drun-vendor.tar.gz/spki
error: could not compile `wasmtime-cli` due to previous error
   Vendoring ssh2 v0.9.3 (/private/tmp/nix-build-drun-vendor.tar.gz.drv-0/rs/cargo-home.iH4/registry/src/github.com-1ecc6299db9ec823/ssh2-0.9.3) to drun-vendor.tar.gz/ssh2
error: builder for '/nix/store/6shmcgg9n1n0l7c0sdd6bb170d2966w0-wasmtime-0.21.0.drv' failed with exit code 101;
       last 10 log lines:
       >                 yanix::sys::bsd::dir::iter_impl::h2f2c32a20989b80a in libyanix-699c0fa6ed60634e.rlib(yanix-699c0fa6ed60634e.yanix.993e2a33-cgu.12.rcgu.o)
       >             "_fstat$INODE64", referenced from:
       >                 yanix::file::fstat::h2c7cb40e007c0839 in libyanix-699c0fa6ed60634e.rlib(yanix-699c0fa6ed60634e.yanix.993e2a33-cgu.0.rcgu.o)
       >             "_fstatat$INODE64", referenced from:
       >                 yanix::file::fstatat::had9f7a12369ea7f3 in libwasi_common-9bd597a51adbffdc.rlib(wasi_common-9bd597a51adbffdc.wasi_common.a51b261f-cgu.6.rcgu.o)
       >           ld: symbol(s) not found for architecture arm64
       >           clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
       >           
       >
       > error: could not compile `wasmtime-cli` due to previous error
       For full logs, run 'nix log /nix/store/6shmcgg9n1n0l7c0sdd6bb170d2966w0-wasmtime-0.21.0.drv'.
error: build of '/nix/store/04wl6h47i7xmn4l8l195xlpnjv0a718w-ocaml4.12.0-uucp-14.0.0.drv', '/nix/store/3vli2xzc16vc7m4fpx968nhy9dvmpadn-ocaml4.12.0-ocamlformat-0.19.0.drv', '/nix/store/48gwh4k52881xi07v2w7fxzx5pddiwlq-drun.drv', '/nix/store/6shmcgg9n1n0l7c0sdd6bb170d2966w0-wasmtime-0.21.0.drv', '/nix/store/ccr88y8wkar8qfpij4i4vkq1lha7slcp-ic-ref-run.drv', '/nix/store/f5mrcazzd4ax7ih63a1gh90zfcvhwqxp-niv-0.2.19.drv', '/nix/store/h8srnrkcni5xjkyamh5w6a9f2ysh834p-qc-motoko-1.drv', '/nix/store/jv1fc3z0rki4z1d6as3pipqcwfyfgfw0-rust-1.59.0-nightly-2021-12-01-48a5999fc.drv', '/nix/store/wfjvsz5viwbmxnx2b9frr4iyrdqhcka0-wasm-profiler-instrument.drv' failed

[paulyoung]

I suspect a newer version of wasmtime will work.

@ggreif, it looks like you may have been trying to do that via https://github.com/NixOS/nixpkgs/pull/109571. Do you have any suggestions?

[paulyoung]

That version of wasmtime does appear to help although I'm seeing some test failures for it. Trying to address those as well.

[paulyoung]

It looks like I can work around those with the export HOME=$TMP; trick.

Now running into https://github.com/nmattia/niv/issues/332

[paulyoung]

With the following changes I was successfully able to enter a nix-shell:

diff --git a/nix/default.nix b/nix/default.nix
index 70f10a5a..187928ff 100644
--- a/nix/default.nix
+++ b/nix/default.nix
@@ -88,6 +88,46 @@ let
           };
         })

+        (self: super: {
+          # https://github.com/nmattia/niv/issues/332#issuecomment-958449218
+          niv = self.haskell.lib.compose.overrideCabal (drv: { enableSeparateBinOutput = false; }) super.haskellPackages.niv;
+
+          # https://github.com/NixOS/nixpkgs/pull/109571
+          wasmtime = with self; rustPlatform.buildRustPackage rec {
+            pname = "wasmtime";
+            version = "0.32.0";
+
+            src = fetchFromGitHub {
+              owner = "bytecodealliance";
+              repo = pname;
+              rev = "v${version}";
+              sha256 = "sha256-iko2G2cUIQYv7Sia8fLtb7d6XCbpOKz31ST62eE19B0";
+              fetchSubmodules = true;
+            };
+
+            cargoSha256 = "sha256-z8x004BbRWi9cRf2I27uiFuu2Jnr1Z3Ey992S5hdyNs";
+
+            nativeBuildInputs = [ python cmake clang ];
+            buildInputs = [ llvmPackages.libclang ] ++
+            lib.optionals stdenv.isDarwin [ darwin.apple_sdk.frameworks.Security ];
+            LIBCLANG_PATH = "${llvmPackages.libclang.lib}/lib";
+
+            configurePhase = ''
+              export HOME=$TMP;
+            '';
+
+            doCheck = true;
+
+            meta = with lib; {
+              description = "Standalone JIT-style runtime for WebAssembly, using Cranelift";
+              homepage = "https://github.com/bytecodealliance/wasmtime";
+              license = licenses.asl20;
+              maintainers = [ maintainers.matthewbauer ];
+              platforms = platforms.unix;
+            };
+          };
+        })
+
       ];
     };
 in
diff --git a/nix/sources.json b/nix/sources.json
index 0eec0718..31f4acbc 100644
--- a/nix/sources.json
+++ b/nix/sources.json
@@ -97,16 +97,16 @@
         "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
     },
     "nixpkgs": {
-        "branch": "release-21.11",
+        "branch": "backport-153630-to-release-21.11",
         "builtin": true,
         "description": "Nixpkgs/NixOS branches that track the Nixpkgs/NixOS channels",
         "homepage": null,
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "d1e59cfc49961e121583abe32e2f3db1550fbcff",
-        "sha256": "03ldf1dlxqf3g8qh9x5vp6vd9zvvr481fyjds111imll69y60wpm",
+        "rev": "17de6590083e80981c4459354530ca6411de485b",
+        "sha256": "05fvgqh4j01wfriqzrhfsaf3sb4knv14gmj1682f88xjfbl4qwgi",
         "type": "tarball",
-        "url": "https://github.com/NixOS/nixpkgs/archive/d1e59cfc49961e121583abe32e2f3db1550fbcff.tar.gz",
+        "url": "https://github.com/NixOS/nixpkgs/archive/17de6590083e80981c4459354530ca6411de485b.tar.gz",
         "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
     },
     "nixpkgs-mozilla": {

[ggreif]

@paulyoung Great that you can use nix-shell on M1! And also thanks for the detective work you have invested! Indeed the Wasmtime package needs some love. It has several problems:

• utimensat is needed by the build, but not available by Darwin < 10.13
• your mentioned test failure (I'll integrate your workaround)
• test wants to access the internet (do you have pure = false in your config?)

So don't hold your breath just yet :-)

[paulyoung]

For what it’s worth, I learned the HOME=$TMP trick a while ago from here to work around the issue reported there: https://github.com/NixOS/nixpkgs/issues/16144#issuecomment-225422439

I don’t think I have pure = false in my config. It’s possible I ran nix-shell --impure but I don’t remember doing that.

[paulyoung]

I've come back to this after a while and now I'm somehow seeing a different wasmtime error:

error: could not compile `wasmtime-cli` due to previous error
error: builder for '/nix/store/3qkdh55ydxi4d97gq6qgrrc4s92hwj28-wasmtime-0.32.0.drv' failed with exit code 101;
       last 10 log lines:
       >   = note: Undefined symbols for architecture x86_64:
       >             "_futimens", referenced from:
       >                 rustix::imp::libc::fs::syscalls::futimens::h40adb1a14bbd1f4a in librustix-6124166338298bee.rlib(rustix-6124166338298bee.rustix.56f6ede2-cgu.5.rcgu.o)
       >             "_utimensat", referenced from:
       >                 rustix::imp::libc::fs::syscalls::utimensat::h67ad5e8dfbb603b0 in librustix-6124166338298bee.rlib(rustix-6124166338298bee.rustix.56f6ede2-cgu.5.rcgu.o)
       >           ld: symbol(s) not found for architecture x86_64
       >           clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
       >
       >
       > error: could not compile `wasmtime-cli` due to previous error
       For full logs, run 'nix log /nix/store/3qkdh55ydxi4d97gq6qgrrc4s92hwj28-wasmtime-0.32.0.drv'.
error: build of '/nix/store/3qkdh55ydxi4d97gq6qgrrc4s92hwj28-wasmtime-0.32.0.drv', '/nix/store/4phzhm3klqygs0zgvdm8jb4z4abk17z7-ic-ref-run.drv', '/nix/store/9n49s3szs960fn7mrgfl7zzgw2zmlb4f-drun.drv', '/nix/store/kb759lqwh0h8rbjfwczb9xciszmxl2gg-niv-0.2.19.drv', '/nix/store/pxbdp5gz8hjjdyzf6mywxpc5mj074rph-rust-1.59.0-nightly-2021-12-01-48a5999fc.drv' failed

[domwoe]

I'm getting the same error trying to install moc with nix-env -i -f . -A moc on a M1.

[sunfishcode]

With https://github.com/bytecodealliance/rustix/pull/275, rustix now works on macOS versions that lack utimensat and futimens. https://github.com/bytecodealliance/wasmtime/pull/4022 is a PR to update Wasmtime.

[paulyoung]

Related: https://github.com/NixOS/nixpkgs/pull/166965

[ninegua]

I was able to build moc binaries on aarch64 with a simpler nix file. It has been tested with nix 2.8.0 and unstable channel on a 2021 m1 mbp running OS X 12.3.1.

https://github.com/ninegua/motoko/commit/ba72f957425d25f95f9de236dcab10fe36246b37

In particular, I think the nix configuration should let users choose their own nixpkgs to avoid building "the world".

[nomeata]

In particular, I think the nix configuration should let users choose their own nixpkgs to avoid building "the world".

As an option, certainly. But by default we do want to pin nixpkgs, right?

[ninegua]

But by default we do want to pin nixpkg

Yes of course

[ninegua]

Let me elaborate a bit. On CI I think it is always good to pin a nixpkg version, do static build, cross compiling, etc etc. On a dev's machine, having the exact setup as CI should be optional, because it is rarely needed, especially when native platform differs.

I spent some time yesterday on this because I wanted to have a fully working nix-shell of https://github.com/dfinity/ic on my new work laptop (m1 MBP). It was a lot simpler to use rustup for that purpose, but for testing, I need to pull in dependencies which include the motoko repo. So I spent some time to get nix-build working on m1. I might spend a bit more time to clean up a few things and upstream my patches (and also to get dfx compiling too) but no guarantees...

[nomeata]

On a dev's machine, having the exact setup as CI should be optional, because it is rarely needed, especially when native platform differs.

I’d say, in that case CI should also run that particular native platform :-)

Especially with things like using a specific nightly or even patching this I think the default should be that nix-shell uses exactly what’s on CI, and only if you know what you are doing (and why) you should be given a way to override that choice. At least so far the Motoko team seemed reasonably happy with that patronizing approach :-)

[ninegua]

Normally I'd agree with you, but buildRustPackage is such a pain... unless somebody solves the incremental build problem for using rust in nix, I'd prefer a very impure dev environment like rustup

[nomeata]

Note that we do have a separate (make- and/or cargo based) incremental build setup where nix only provides the tools. We are not using nix-build for incremental work.

[paulyoung]

@ninegua I tried your branch and ran into this:

building '/nix/store/jggfr9py5nscx5y9843y80kzp0krmys5-git-ls-files.drv'...
error: Package ‘libcxx-13.0.0’ in /nix/store/jsciqap989svc3a0bf2gjnsh38k4shw9-nixpkgs-22.05pre335126.391f93a83c3/nixpkgs/pkgs/development/compilers/llvm/13/libcxx/default.nix:42 is marked as broken, refusing to evaluate.

       a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowBroken = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowBroken = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)

I tried again with NIXPKGS_ALLOW_BROKEN=1 but got what appears to be a related failure.

[paulyoung]

Which version of nixpkgs are you using?

[ninegua]

@paulyoung For m1 OS X, you have to switch to use unstable channel, which has wasmtime 0.36.0. For m1 Linux, you need to stay with the 21.11 channel, which has wasmtime 0.21.0. It is unfortunate that 0.21.0 panics in wasmtime testing on m1 OS X, while 0.36.0 panics in testing on m1 Linux.

Edit: If your failure if related to NIXPKGS_ALLOW_BROKEN, then you should also add --impure to nix-build. For nix-shell it should not be necessary.

Also, wasmtime is not needed for compilation, so if it gets in the way, you can comment it out from rts/Makefile.

[ninegua]

I was able to build all binaries on aarch64 OS X and Linux. You can try it out https://github.com/ninegua/ic-nix