Trouble with 0.5.24 on NixOS

[bbarker]

I'm not immediately sure what may have caused this change in behavior.

error: builder for '/nix/store/3l3r4zjcr30g9z7scjdr8zvkvw8p0707-unison-code-manager-0.5.24.drv' failed with exit code 1;
       last 10 log lines:
       > patching script interpreter paths in /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24
       > calling 'fixupOutput' function hook '_pruneLibtoolFiles'
       > calling 'fixupOutput' function hook '_doStrip'
       > stripping (with command strip and flags -S -p) in  /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/lib /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin
       > evaling implicit 'postFixup' string hook
       > /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm: line 51: /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/unison/unison: cannot execute: required file not found
       > /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm: line 51: /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/unison/unison: cannot execute: required file not found
       > /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm: line 51: /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/unison/unison: cannot execute: required file not found
       > installShellCompletion: error: installed shell completion file `/nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/share/bash-completion/completions/ucm.bash' does not exist or has zero size
       > /nix/store/5r0df66ikad3xw06azlqvswcvncll8wa-stdenv-linux/setup: line 193: pop_var_context: head of shell_variables not a function context
       For full logs, run 'nix log /nix/store/3l3r4zjcr30g9z7scjdr8zvkvw8p0707-unison-code-manager-0.5.24.drv'.
[0/8 built (1 failed), 5/2/8 copied (290.5/373.9 MiB), 85.4/118.2 MiB DL] building unison-code-[0/8 built (1 failed), 5/2/8 copied (291.4/373.9 MiB), 85.6/118.2 MiB DL] building unison-code-error: 1 dependencies of derivation '/nix/store/5rc231ps7hvv6ysvlz4fjl68nyyrvfwb-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/wqrlkvdj7701i97k5w1in086isrzy8h6-home-manager-generation.drv' failed to build
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   .config/home-manager/flake.nix
    modified:   flake_locks/flake-x86_64-Linux-nixos.lock

no changes added to commit (use "git add" and/or "git commit -a")

[bbarker@nixos:~/workspace/dotfiles]$ ls /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm
/nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm

[bbarker@nixos:~/workspace/dotfiles]$ ls /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin
ucm

[bbarker@nixos:~/workspace/dotfiles]$ /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm
Could not start dynamically linked executable: /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/unison/unison
NixOS cannot run dynamically linked executables intended for generic
linux environments out of the box. For more information, see:
https://nix.dev/permalink/stub-ld

[ceedubs]

Thanks for the report. Strangely I'm on NixOS and I'm not seeing the same with ucm installed via home-manager in a flake. Are your dotfiles public somewhere that I could take a look @bbarker?

[bbarker]

That is odd. I added an PR here that has the updated versions listed:

https://github.com/bbarker/dotfiles/pull/4/files

I added ai-sh but I wouldn't think that would matter

[ceedubs]

Looking at that diff it looks like you might have updated straight from 0.5.21 to 0.5.24. Does that sound right? I suspect that 0.5.23 is where the trouble started for you, since that is the one that turned ucm into a wrapper script. But I'm still not sure why you encounter this issue and I don't. I'll poke around a bit more.

[bbarker]

Thanks - You're right that I likely did not have any updates since 0.5.21. This is on NixOS 24.05, which I had also updated to recently, before trying UCM 0.5.23.

I can also try doing this on a non-NixOS system and see if the issue exists there as well.

On Mon, Jul 22, 2024 at 10:39 AM Cody Allen @.***> wrote:

Looking at that diff it looks like you might have updated straight from 0.5.21 to 0.5.24. Does that sound right? I suspect that 0.5.23 is where the trouble started for you, since that is the one that turned ucm into a wrapper script. But I'm still not sure why you encounter this issue and I don't. I'll poke around a bit more.

— Reply to this email directly, view it on GitHub https://github.com/ceedubs/unison-nix/issues/97#issuecomment-2243128197, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG7XDVJC5I3F6RADISLY3DZNUKSJAVCNFSM6AAAAABLCRXJA6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGEZDQMJZG4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Brandon Barker @.***

[ceedubs]

The strange thing is that I am also on NixOS 24.05 with a similar flake setup. I'm struggling to think of why we are encountering different results. If only we had a deterministic OS configured declaratively!

[ceedubs]

Hmm here is something suspicious. I have a different hash for unison-code-manager-0.5.24:

❯ ls /nix/store/l881g8z7az2n7d4k68q887pc0j9fabw0-unison-code-manager-0.5.24
bin/  lib/  share/  ui/  unison/

I wonder if some derivation somewhere had that configured with the wrong hash and one of us is hitting a "oh I already have that hash I won't fetch it" error or something?

[bbarker]

Interesting - I'm not sure I totally understand how that could happen in Nix, as I thought hashes were always checked deterministically. That said, if there were different build inputs, then it could happen, which I guess could occur due to slightly different versions of nixpkgs being used in each case (that seems to be the only depenency of unison)?

I tried it on my most different system (MacOS), and have the same issue:

-sh-3.2$ ./build.sh
Already up to date.
Looking for /Users/bbarker/.config/home-manager/home-x86_64-Darwin-C02FD66VMD6M.nix
Found home-x86_64-Darwin-C02FD66VMD6M.nix
Substitution complete. The Nix system is now set to x86_64-darwin.
home-manager is available in the PATH; updating.
this derivation will be built:
  /nix/store/0bdlf42216kqg3gn3d7yrwvbsd5y12g8-home-manager.drv
building '/nix/store/0bdlf42216kqg3gn3d7yrwvbsd5y12g8-home-manager.drv'...
unpacking channels...
warning: updating lock file '/Users/bbarker/.config/home-manager/flake.lock':
• Updated input 'home-manager':
    'github:nix-community/home-manager/d7830d05421d0ced83a0f007900898bdcaf2a2ca' (2024-06-19)
  → 'github:nix-community/home-manager/635563f245309ef5320f80c7ebcb89b2398d2949' (2024-07-21)
• Updated input 'nixpkgs':
    'github:nixos/nixpkgs/c00d587b1a1afbf200b1d8f0b0e4ba9deb1c7f0e' (2024-06-18)
  → 'github:nixos/nixpkgs/1d9c2c9b3e71b9ee663d11c5d298727dace8d374' (2024-07-19)
• Updated input 'unison-lang':
    'github:ceedubs/unison-nix/923749a56cb1b36291b420c53d49f180d992c304' (2024-06-05)
  → 'github:ceedubs/unison-nix/18a9798ed7a1acd69b3fd525093c2723854b4664' (2024-07-13)
error: builder for '/nix/store/gp7qqraldssaiz07qddyirmn60k2iw2a-unison-code-manager-0.5.24.drv' failed with exit code 1;
       last 10 log lines:
       >  e     s  e r  v e    r   w  i  t h  o  u t    t h ceo  d e b ca os mm e a
       > cn od d-heleaidbnela
       >  ese s
       > s              h  e   a   d    l   e   s Rs   u    n s   t  h e    c  o i d e nbat es  er   sfReurnvears w itcthhee.o
       > cutod ebtaseh ese rcvoemrm waitnhdou-tli nte
       > h  e        c o m m a nd -l i ne
       >                        in t  er f a c e .
       >      interface.
       > installShellCompletion: error: installed shell completion file `/nix/store/anvlsdpwbx28dgzx442dn0hgbwaw4vpl-unison-code-manager-0.5.24/share/bash-completion/completions/ucm.bash' does not exist or has zero size
       > /nix/store/x4lrdlck6skb3z9cwaz8mzga781yl90s-stdenv-darwin/setup: line 202: pop_var_context: head of shell_variables not a function context
       For full logs, run 'nix log /nix/store/gp7qqraldssaiz07qddyirmn60k2iw2a-unison-code-manager-0.5.24.drv'.
error: 1 dependencies of derivation '/nix/store/l23j3k09paakxsjgazlh4wnw7h8wq3hz-home-manager-applications.drv' failed to build
error: 1 dependencies of derivation '/nix/store/gcyi5frqi3xk870kyvqsvfwcfxm7k9pv-home-manager-fonts.drv' failed to build
error: 1 dependencies of derivation '/nix/store/dcjl0dfmh7y5kvfvqqhr9k7y7dr7hm4g-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/aqr1lrrpcgcpydpbf4yfvmj2b92ymnly-home-manager-generation.drv' failed to build
On branch update_and_add_ai-sh
Your branch is up to date with 'origin/update_and_add_ai-sh'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   flake_locks/flake-x86_64-Darwin-C02FD66VMD6M.lock

-sh-3.2$ ls -d /nix/store/*unison-code-manager-0.5.24
/nix/store/anvlsdpwbx28dgzx442dn0hgbwaw4vpl-unison-code-manager-0.5.24

[ceedubs]

I'm not sure I totally understand how that could happen in Nix, as I thought hashes were always checked deterministically.

The scenario that I was thinking of (and have hit before) is when a derivation calls fetchurl (or something similar) and provides an expected hash. If the expected hash is already in the nix store then nix won't perform the URL fetch. So if you accidentally forget to update the sha256 field of the fetchurl call you might not get the new version after updating the URL. It's possible that I goofed at some point.

0.5.25 came out yesterday and I've updated this repo to it. Do you still see the same issue there?

[bbarker]

It seems like the issue persists for me in 0.5.25:

error: builder for '/nix/store/zq1ixqbnimvqwhmmk5sgzldm4rw3pb5s-unison-code-manager-0.5.25.drv' failed with exit code 1;
       last 10 log lines:
       > patching script interpreter paths in /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25
       > calling 'fixupOutput' function hook '_pruneLibtoolFiles'
       > calling 'fixupOutput' function hook '_doStrip'
       > stripping (with command strip and flags -S -p) in  /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/lib /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin
       > evaling implicit 'postFixup' string hook
       > /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm: line 51: /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/unison: cannot execute: required file not found
       > /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm: line 51: /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/unison: cannot execute: required file not found
       > /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm: line 51: /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/unison: cannot execute: required file not found
       > installShellCompletion: error: installed shell completion file `/nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/share/bash-completion/completions/ucm.bash' does not exist or has zero size
       > /nix/store/5r0df66ikad3xw06azlqvswcvncll8wa-stdenv-linux/setup: line 193: pop_var_context: head of shell_variables not a function context
       For full logs, run 'nix log /nix/store/zq1ixqbnimvqwhmmk5sgzldm4rw3pb5s-unison-code-manager-0.5.25.drv'.
error: 1 dependencies of derivation '/nix/store/hglnqxfzc0183y2ah2b8xzghva9hy33i-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/mg5fjsv48i338nk4zcwm7xz4wjddv6c7-home-manager-generation.drv' failed to build
On branch update_and_add_ai-sh

Here are the changes since yesterday when we tried 0.5.24:

--- a/flake_locks/flake-x86_64-Linux-nixos.lock
+++ b/flake_locks/flake-x86_64-Linux-nixos.lock
@@ -25,11 +25,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1721534365,
-        "narHash": "sha256-XpZOkaSJKdOsz1wU6JfO59Rx2fqtcarQ0y6ndIOKNpI=",
+        "lastModified": 1721714663,
+        "narHash": "sha256-ZDW5+rlROxaOuoEfIQM7Gqhoa+WALEYdYIiZhyJjAu0=",
         "owner": "nix-community",
         "repo": "home-manager",
-        "rev": "635563f245309ef5320f80c7ebcb89b2398d2949",
+        "rev": "7560dc942a6fbd37ebd1310b3dbda513de2d4b82",
         "type": "github"
       },
       "original": {
@@ -62,11 +62,11 @@
     },
     "nixpkgs": {
       "locked": {
-        "lastModified": 1721379653,
-        "narHash": "sha256-8MUgifkJ7lkZs3u99UDZMB4kbOxvMEXQZ31FO3SopZ0=",
+        "lastModified": 1721562059,
+        "narHash": "sha256-Tybxt65eyOARf285hMHIJ2uul8SULjFZbT9ZaEeUnP8=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "1d9c2c9b3e71b9ee663d11c5d298727dace8d374",
+        "rev": "68c9ed8bbed9dfce253cc91560bf9043297ef2fe",
         "type": "github"
       },
       "original": {
@@ -124,11 +124,11 @@
         "unison": "unison"
       },
       "locked": {
-        "lastModified": 1720865364,
-        "narHash": "sha256-ZmIcRUh7DKZFW6Y4fl08waJYXKk21PG7KWzwUgd+3Nc=",
+        "lastModified": 1721739307,
+        "narHash": "sha256-KGbUQYy15W820QnQNmcOWuV7q7GJS9jrg1Zdyz3Co7o=",
         "owner": "ceedubs",
         "repo": "unison-nix",
-        "rev": "18a9798ed7a1acd69b3fd525093c2723854b4664",
+        "rev": "3f10238cfbdc6395ef0b56d8c6e659585df13d6a",
         "type": "github"
       },
       "original": {

Barring any other ideas, I may try to find a system to try it on that isn't using my home-manager setup to see if that is somehow an issue, though as it was working previously, if that is the case, I guess it must be some interaction with the addition of the wrapper scripts, but again, just guessing.

[bbarker]

To follow up on my last post, I do see the following at the specified nix store entry:

ls /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm
/nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm

$ ls /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/*
/nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/unison

So the unison file is there, I think, and looks like an executable (not a script). Could something be happening out of sequence to cause the error?

[ceedubs]

I just tried from macos with home-manager and I don't see an issue there either. Strange. I was wondering if it could be that an overlay is getting applied out of order or something, but then I don't think that you would have unison-code-manager-0.5.25. I'm a bit stumped.

[ceedubs]

Oh one other thing that I notice is that you are on unstable nixpkgs while I am on 24.05. I wonder if that could be relevant?

[bbarker]

Oh one other thing that I notice is that you are on unstable nixpkgs while I am on 24.05. I wonder if that could be relevant?

This was a good callout; I tried 24.05 on MacOS today and no issues. So now I guess we need to think about why unstable broke things.

https://github.com/bbarker/dotfiles/commit/7f1981309aa1f41d641bb050715e88fe2ea269ef

[ceedubs]

I still don't know quite what changed with unstable, but I have a better idea of what's going on here. The ucm derivation wraps a prebuilt binary that was not built via Nix, which goes against the grain of Nix builds. Something in unstable seems to be making this finally complain.

Here is a blog post that is relevant.

I'll try to dig deeper at some point if nobody beats me to it. Though perhaps we should just be building Unison from source in the derivation?

[bbarker]

That's a nice blog post.

Building from source sounds good in principle, though it might be best to use cachix in that scenario. I've only ever consumed from it that I remember, but it appears public caches should be free:

https://nixos.wiki/wiki/Binary_Cache

[ceedubs]

@bbarker I'm hoping that this is fixed by https://github.com/ceedubs/unison-nix/pull/103 . Are you able to verify?

[ceedubs]

I think that this is resolved. @bbarker feel free to reopen if I'm mistaken.