search

ceedubs / unison-nix

Nix support for the Unison programming language
MIT License
56 stars 8 forks source link

Trouble with 0.5.24 on NixOS #97

Open bbarker opened 1 month ago

bbarker commented 1 month ago

I'm not immediately sure what may have caused this change in behavior.

error: builder for '/nix/store/3l3r4zjcr30g9z7scjdr8zvkvw8p0707-unison-code-manager-0.5.24.drv' failed with exit code 1;
       last 10 log lines:
       > patching script interpreter paths in /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24
       > calling 'fixupOutput' function hook '_pruneLibtoolFiles'
       > calling 'fixupOutput' function hook '_doStrip'
       > stripping (with command strip and flags -S -p) in  /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/lib /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin
       > evaling implicit 'postFixup' string hook
       > /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm: line 51: /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/unison/unison: cannot execute: required file not found
       > /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm: line 51: /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/unison/unison: cannot execute: required file not found
       > /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm: line 51: /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/unison/unison: cannot execute: required file not found
       > installShellCompletion: error: installed shell completion file `/nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/share/bash-completion/completions/ucm.bash' does not exist or has zero size
       > /nix/store/5r0df66ikad3xw06azlqvswcvncll8wa-stdenv-linux/setup: line 193: pop_var_context: head of shell_variables not a function context
       For full logs, run 'nix log /nix/store/3l3r4zjcr30g9z7scjdr8zvkvw8p0707-unison-code-manager-0.5.24.drv'.
[0/8 built (1 failed), 5/2/8 copied (290.5/373.9 MiB), 85.4/118.2 MiB DL] building unison-code-[0/8 built (1 failed), 5/2/8 copied (291.4/373.9 MiB), 85.6/118.2 MiB DL] building unison-code-error: 1 dependencies of derivation '/nix/store/5rc231ps7hvv6ysvlz4fjl68nyyrvfwb-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/wqrlkvdj7701i97k5w1in086isrzy8h6-home-manager-generation.drv' failed to build
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   .config/home-manager/flake.nix
    modified:   flake_locks/flake-x86_64-Linux-nixos.lock

no changes added to commit (use "git add" and/or "git commit -a")

[bbarker@nixos:~/workspace/dotfiles]$ ls /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm
/nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm

[bbarker@nixos:~/workspace/dotfiles]$ ls /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin
ucm

[bbarker@nixos:~/workspace/dotfiles]$ /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/bin/ucm
Could not start dynamically linked executable: /nix/store/78wb1vvsvrcm5hs5zmsp60zckpf1dq2g-unison-code-manager-0.5.24/unison/unison
NixOS cannot run dynamically linked executables intended for generic
linux environments out of the box. For more information, see:
https://nix.dev/permalink/stub-ld
ceedubs commented 1 month ago

Thanks for the report. Strangely I'm on NixOS and I'm not seeing the same with ucm installed via home-manager in a flake. Are your dotfiles public somewhere that I could take a look @bbarker?

bbarker commented 1 month ago

That is odd. I added an PR here that has the updated versions listed:

https://github.com/bbarker/dotfiles/pull/4/files

I added ai-sh but I wouldn't think that would matter

ceedubs commented 1 month ago

Looking at that diff it looks like you might have updated straight from 0.5.21 to 0.5.24. Does that sound right? I suspect that 0.5.23 is where the trouble started for you, since that is the one that turned ucm into a wrapper script. But I'm still not sure why you encounter this issue and I don't. I'll poke around a bit more.

bbarker commented 1 month ago

Thanks - You're right that I likely did not have any updates since 0.5.21. This is on NixOS 24.05, which I had also updated to recently, before trying UCM 0.5.23.

I can also try doing this on a non-NixOS system and see if the issue exists there as well.

On Mon, Jul 22, 2024 at 10:39 AM Cody Allen @.***> wrote:

Looking at that diff it looks like you might have updated straight from 0.5.21 to 0.5.24. Does that sound right? I suspect that 0.5.23 is where the trouble started for you, since that is the one that turned ucm into a wrapper script. But I'm still not sure why you encounter this issue and I don't. I'll poke around a bit more.

— Reply to this email directly, view it on GitHub https://github.com/ceedubs/unison-nix/issues/97#issuecomment-2243128197, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG7XDVJC5I3F6RADISLY3DZNUKSJAVCNFSM6AAAAABLCRXJA6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGEZDQMJZG4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Brandon Barker @.***

ceedubs commented 1 month ago

The strange thing is that I am also on NixOS 24.05 with a similar flake setup. I'm struggling to think of why we are encountering different results. If only we had a deterministic OS configured declaratively!

ceedubs commented 1 month ago

Hmm here is something suspicious. I have a different hash for unison-code-manager-0.5.24: 

❯ ls /nix/store/l881g8z7az2n7d4k68q887pc0j9fabw0-unison-code-manager-0.5.24
bin/  lib/  share/  ui/  unison/

I wonder if some derivation somewhere had that configured with the wrong hash and one of us is hitting a "oh I already have that hash I won't fetch it" error or something?

bbarker commented 1 month ago

Interesting - I'm not sure I totally understand how that could happen in Nix, as I thought hashes were always checked deterministically. That said, if there were different build inputs, then it could happen, which I guess could occur due to slightly different versions of nixpkgs being used in each case (that seems to be the only depenency of unison)?

I tried it on my most different system (MacOS), and have the same issue:

-sh-3.2$ ./build.sh
Already up to date.
Looking for /Users/bbarker/.config/home-manager/home-x86_64-Darwin-C02FD66VMD6M.nix
Found home-x86_64-Darwin-C02FD66VMD6M.nix
Substitution complete. The Nix system is now set to x86_64-darwin.
home-manager is available in the PATH; updating.
this derivation will be built:
  /nix/store/0bdlf42216kqg3gn3d7yrwvbsd5y12g8-home-manager.drv
building '/nix/store/0bdlf42216kqg3gn3d7yrwvbsd5y12g8-home-manager.drv'...
unpacking channels...
warning: updating lock file '/Users/bbarker/.config/home-manager/flake.lock':
• Updated input 'home-manager':
    'github:nix-community/home-manager/d7830d05421d0ced83a0f007900898bdcaf2a2ca' (2024-06-19)
  → 'github:nix-community/home-manager/635563f245309ef5320f80c7ebcb89b2398d2949' (2024-07-21)
• Updated input 'nixpkgs':
    'github:nixos/nixpkgs/c00d587b1a1afbf200b1d8f0b0e4ba9deb1c7f0e' (2024-06-18)
  → 'github:nixos/nixpkgs/1d9c2c9b3e71b9ee663d11c5d298727dace8d374' (2024-07-19)
• Updated input 'unison-lang':
    'github:ceedubs/unison-nix/923749a56cb1b36291b420c53d49f180d992c304' (2024-06-05)
  → 'github:ceedubs/unison-nix/18a9798ed7a1acd69b3fd525093c2723854b4664' (2024-07-13)
error: builder for '/nix/store/gp7qqraldssaiz07qddyirmn60k2iw2a-unison-code-manager-0.5.24.drv' failed with exit code 1;
       last 10 log lines:
       >  e     s  e r  v e    r   w  i  t h  o  u t    t h ceo  d e b ca os mm e a
       > cn od d-heleaidbnela
       >  ese s
       > s              h  e   a   d    l   e   s Rs   u    n s   t  h e    c  o i d e nbat es  er   sfReurnvears w itcthhee.o
       > cutod ebtaseh ese rcvoemrm waitnhdou-tli nte
       > h  e        c o m m a nd -l i ne
       >                        in t  er f a c e .
       >      interface.
       > installShellCompletion: error: installed shell completion file `/nix/store/anvlsdpwbx28dgzx442dn0hgbwaw4vpl-unison-code-manager-0.5.24/share/bash-completion/completions/ucm.bash' does not exist or has zero size
       > /nix/store/x4lrdlck6skb3z9cwaz8mzga781yl90s-stdenv-darwin/setup: line 202: pop_var_context: head of shell_variables not a function context
       For full logs, run 'nix log /nix/store/gp7qqraldssaiz07qddyirmn60k2iw2a-unison-code-manager-0.5.24.drv'.
error: 1 dependencies of derivation '/nix/store/l23j3k09paakxsjgazlh4wnw7h8wq3hz-home-manager-applications.drv' failed to build
error: 1 dependencies of derivation '/nix/store/gcyi5frqi3xk870kyvqsvfwcfxm7k9pv-home-manager-fonts.drv' failed to build
error: 1 dependencies of derivation '/nix/store/dcjl0dfmh7y5kvfvqqhr9k7y7dr7hm4g-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/aqr1lrrpcgcpydpbf4yfvmj2b92ymnly-home-manager-generation.drv' failed to build
On branch update_and_add_ai-sh
Your branch is up to date with 'origin/update_and_add_ai-sh'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   flake_locks/flake-x86_64-Darwin-C02FD66VMD6M.lock
-sh-3.2$ ls -d /nix/store/*unison-code-manager-0.5.24
/nix/store/anvlsdpwbx28dgzx442dn0hgbwaw4vpl-unison-code-manager-0.5.24
ceedubs commented 1 month ago

I'm not sure I totally understand how that could happen in Nix, as I thought hashes were always checked deterministically.

The scenario that I was thinking of (and have hit before) is when a derivation calls fetchurl (or something similar) and provides an expected hash. If the expected hash is already in the nix store then nix won't perform the URL fetch. So if you accidentally forget to update the sha256 field of the fetchurl call you might not get the new version after updating the URL. It's possible that I goofed at some point.

0.5.25 came out yesterday and I've updated this repo to it. Do you still see the same issue there?

bbarker commented 1 month ago

It seems like the issue persists for me in 0.5.25:

error: builder for '/nix/store/zq1ixqbnimvqwhmmk5sgzldm4rw3pb5s-unison-code-manager-0.5.25.drv' failed with exit code 1;
       last 10 log lines:
       > patching script interpreter paths in /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25
       > calling 'fixupOutput' function hook '_pruneLibtoolFiles'
       > calling 'fixupOutput' function hook '_doStrip'
       > stripping (with command strip and flags -S -p) in  /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/lib /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin
       > evaling implicit 'postFixup' string hook
       > /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm: line 51: /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/unison: cannot execute: required file not found
       > /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm: line 51: /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/unison: cannot execute: required file not found
       > /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm: line 51: /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/unison: cannot execute: required file not found
       > installShellCompletion: error: installed shell completion file `/nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/share/bash-completion/completions/ucm.bash' does not exist or has zero size
       > /nix/store/5r0df66ikad3xw06azlqvswcvncll8wa-stdenv-linux/setup: line 193: pop_var_context: head of shell_variables not a function context
       For full logs, run 'nix log /nix/store/zq1ixqbnimvqwhmmk5sgzldm4rw3pb5s-unison-code-manager-0.5.25.drv'.
error: 1 dependencies of derivation '/nix/store/hglnqxfzc0183y2ah2b8xzghva9hy33i-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/mg5fjsv48i338nk4zcwm7xz4wjddv6c7-home-manager-generation.drv' failed to build
On branch update_and_add_ai-sh

Here are the changes since yesterday when we tried 0.5.24:

--- a/flake_locks/flake-x86_64-Linux-nixos.lock
+++ b/flake_locks/flake-x86_64-Linux-nixos.lock
@@ -25,11 +25,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1721534365,
-        "narHash": "sha256-XpZOkaSJKdOsz1wU6JfO59Rx2fqtcarQ0y6ndIOKNpI=",
+        "lastModified": 1721714663,
+        "narHash": "sha256-ZDW5+rlROxaOuoEfIQM7Gqhoa+WALEYdYIiZhyJjAu0=",
         "owner": "nix-community",
         "repo": "home-manager",
-        "rev": "635563f245309ef5320f80c7ebcb89b2398d2949",
+        "rev": "7560dc942a6fbd37ebd1310b3dbda513de2d4b82",
         "type": "github"
       },
       "original": {
@@ -62,11 +62,11 @@
     },
     "nixpkgs": {
       "locked": {
-        "lastModified": 1721379653,
-        "narHash": "sha256-8MUgifkJ7lkZs3u99UDZMB4kbOxvMEXQZ31FO3SopZ0=",
+        "lastModified": 1721562059,
+        "narHash": "sha256-Tybxt65eyOARf285hMHIJ2uul8SULjFZbT9ZaEeUnP8=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "1d9c2c9b3e71b9ee663d11c5d298727dace8d374",
+        "rev": "68c9ed8bbed9dfce253cc91560bf9043297ef2fe",
         "type": "github"
       },
       "original": {
@@ -124,11 +124,11 @@
         "unison": "unison"
       },
       "locked": {
-        "lastModified": 1720865364,
-        "narHash": "sha256-ZmIcRUh7DKZFW6Y4fl08waJYXKk21PG7KWzwUgd+3Nc=",
+        "lastModified": 1721739307,
+        "narHash": "sha256-KGbUQYy15W820QnQNmcOWuV7q7GJS9jrg1Zdyz3Co7o=",
         "owner": "ceedubs",
         "repo": "unison-nix",
-        "rev": "18a9798ed7a1acd69b3fd525093c2723854b4664",
+        "rev": "3f10238cfbdc6395ef0b56d8c6e659585df13d6a",
         "type": "github"
       },
       "original": {

Barring any other ideas, I may try to find a system to try it on that isn't using my home-manager setup to see if that is somehow an issue, though as it was working previously, if that is the case, I guess it must be some interaction with the addition of the wrapper scripts, but again, just guessing.

bbarker commented 1 month ago

To follow up on my last post, I do see the following at the specified nix store entry:

ls /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm
/nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/bin/ucm

$ ls /nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/*
/nix/store/h9470smw77kd8dw08yk317kc1qsr21sb-unison-code-manager-0.5.25/unison/unison

So the unison file is there, I think, and looks like an executable (not a script). Could something be happening out of sequence to cause the error?

ceedubs commented 1 month ago

I just tried from macos with home-manager and I don't see an issue there either. Strange. I was wondering if it could be that an overlay is getting applied out of order or something, but then I don't think that you would have unison-code-manager-0.5.25. I'm a bit stumped.

ceedubs commented 1 month ago

Oh one other thing that I notice is that you are on unstable nixpkgs while I am on 24.05. I wonder if that could be relevant?

bbarker commented 1 month ago

Oh one other thing that I notice is that you are on unstable nixpkgs while I am on 24.05. I wonder if that could be relevant?

This was a good callout; I tried 24.05 on MacOS today and no issues. So now I guess we need to think about why unstable broke things.

https://github.com/bbarker/dotfiles/commit/7f1981309aa1f41d641bb050715e88fe2ea269ef

ceedubs commented 1 month ago

I still don't know quite what changed with unstable, but I have a better idea of what's going on here. The ucm derivation wraps a prebuilt binary that was not built via Nix, which goes against the grain of Nix builds. Something in unstable seems to be making this finally complain.

Here is a blog post that is relevant.

I'll try to dig deeper at some point if nobody beats me to it. Though perhaps we should just be building Unison from source in the derivation?

bbarker commented 1 month ago

That's a nice blog post.

Building from source sounds good in principle, though it might be best to use cachix in that scenario. I've only ever consumed from it that I remember, but it appears public caches should be free:

https://nixos.wiki/wiki/Binary_Cache