search

nix-community / emacs-overlay

Bleeding edge emacs overlay [maintainer=@adisbladis]
511 stars 166 forks source link

Recent builds are not available in Cachix #305

Closed joshbode closed 1 year ago

joshbode commented 1 year ago

The recent builds of emacsPgtk (and emacsGit) don't appear to be being pushed to Cachix for nix-community:

$ nix flake metadata emacs-overlay
Resolved URL:  github:nix-community/emacs-overlay
Locked URL:    github:nix-community/emacs-overlay/3571b94216954399d7c18e4b2f6df4372f447497
Description:   Bleeding edge Emacs overlay
Path:          /nix/store/pkpxhw7dk406y2v2y8w50mhqyvgzqzxb-source
Revision:      3571b94216954399d7c18e4b2f6df4372f447497
Last modified: 2023-03-10 05:42:21
Inputs:
├───flake-utils: github:numtide/flake-utils/5aed5285a952e0b949eb3ba02c12fa4fcfef535f
└───nixpkgs: github:NixOS/nixpkgs/0f213d0fee84280d8c3a97f7469b988d6fe5fcdf
$ HASH=$(basename $(nix eval --raw emacs-overlay#emacsPgtk) | cut -d- -f1)
$ echo "${HASH}"
inchc1sl924mcrv6d2k41927qaw6w7rx
$ curl "https://nix-community.cachix.org/${HASH}.narinfo"
curl: (22) The requested URL returned error: 404

Note: emacsUnstable is available, but this is probably because it is based on a tag that hasn't changed for some time:

$ HASH=$(basename $(nix eval --raw emacs-overlay#emacsUnstable) | cut -d- -f1)
$ echo "${HASH}"
f06i229n4qa7gyrxq0z64bwv1ydsnj9s

$ curl "https://nix-community.cachix.org/${HASH}.narinfo"
StorePath: /nix/store/f06i229n4qa7gyrxq0z64bwv1ydsnj9s-emacs-unstable-28.2
URL: nar/9a49625df486ee4f5c54fc1a5ae8567fb632975165b2d3452342dbb08816c84d.nar.xz
Compression: xz
FileHash: sha256:9a49625df486ee4f5c54fc1a5ae8567fb632975165b2d3452342dbb08816c84d
FileSize: 75141152
NarHash: sha256:0ynyb50684s6vjvxwyhn3xynqfv510qnydmpr9zb0pa4pdbw9r9a
NarSize: 297779184
References: ...
Deriver: xpwv539qnsskkk94pldz4spcnn9f7i7g-emacs-unstable-28.2.drv
Sig: nix-community.cachix.org-1:+ZOSwi6phrjJEh9YCWK2DeF/KSHGjca3iZwtMFKz0mvJX7F7An+lI6bjRVJs+b4gjR/qs/o8T7/pSjeGBuMMAA==

This seems like it might be related to this message in the action logs, which has been present for a while:

Post job cleanup.
Cachix: push
  Pushing is disabled as signingKey nor authToken are set (or are emtpy?) in your YAML file.

see the final step "Post Add nix-community cache" : https://github.com/nix-community/emacs-overlay/actions/runs/4376519611/jobs/7658748503

joshbode commented 1 year ago

Hmm - this appears to be a duplicate of #208

joshbode commented 1 year ago

That said, it looks like there probably needs to be a signingKey backed by a secret in the cachix/cachix-action action (currently using v10 but v12 is also available):

    - name: Add nix-community cache
      uses: cachix/cachix-action@v10
      with:
        name: nix-community
        signingKey: "${{ secrets.CACHIX_SIGNING_KEY }}"  # <-- here

compare to these examples:

idlip commented 1 year ago

@joshbode Nobody has responded to this issue so far?

I am not able to fetch the binary cache, whatever method I try (hydra build revision, or with other command from that mentioned issues)

Any workaround or solution to get cache working?

joshbode commented 1 year ago

Apart from just to wait for the long, local build, there's nothing left to do but invoke sibling rivalry :) The Neovim overlay uses authToken instead of signingKey: https://github.com/nix-community/neovim-nightly-overlay/commit/4796a65f887bad090cfe8261a8bccb5b0ebd9251

idlip commented 1 year ago

It is very sad, people have not taken action. Even people on matrix did not respond properly.

I will try to post this once there. These kind of things really start me to hate nix (I was a good arch user lol)

Someone who has emacs path as /nix/store/ynqz17q1hqydbrgd0bcff3zkwgwcmd4d-emacs-pgtk-with-packages-20230406.0/bin/emacs said, he got the binary cache by changing the revision for emacs-overlay.

When i tried it now, rev number according to hydra latest build, I could not get the cache.

joshbode commented 1 year ago

It is very sad, people have not taken action. Even people on matrix did not respond properly.

It's a volunteer effort so these things can take time. I'd usually make a PR but in this case we're dependent on someone who can get a key/token and establish it as a secret for the build job

idlip commented 1 year ago

@joshbode I was able to hit the cache today.

I just happened to check the latest hydra build, and ran curl over that hash

curl https://nix-community.cachix.org/f51ngj5s5qa21n4avcg8gkfsdakn4457.narinfo

And it gave results : 

StorePath: /nix/store/f51ngj5s5qa21n4avcg8gkfsdakn4457-emacs-pgtk-20230412.0
URL: nar/eb204a11f7ceb84ecc22833767dd2235982c31e56df02f66f58f91e435ee200f.nar.zst
Compression: zstd
FileHash: sha256:eb204a11f7ceb84ecc22833767dd2235982c31e56df02f66f58f91e435ee200f
FileSize: 102331624
NarHash: sha256:03gnq6nwm374yghzqxwwr2g6va519viljb76b4zxj9dxnncfww07
NarSize: 324656808
References: 0gpxzb4044hqgrcpshpgaxn2asi2ykdb-librsvg-2.55.1 0hx32wk55ml88jrb1qxwg5c5yazfm6gf-bash-5.2-p15 0zcwhj24c06a6wjp8r4ry4k0rvkzyrib-harfbuzz-7.0.1 2qk14cdp3b2rk9sr3m7swmpsfcyr5i8a-libgccjit-12.2.0 33pkg1f1ad2wh2rkl2c48hf8kfxcjgw5-systemd-253.1 35vld1iywd5iy111pxavsd7iaqs9xbjs-libwebp-1.3.0 37ycsg5dk45kwggjvg9qm996ncn4mh39-libtiff-4.5.0 5fk74drrnrhgmcwxvsnmv2lx1srgdfkp-glib-2.74.5 5gr9s12xibq0c0ydf74b0xy7hb7bzs9y-gsettings-desktop-schemas-43.0 7y04gsisjii9wsxmf36a0d2kv1g72dn1-ncurses-6.4 8abpm1gfm54lb4m539z7f76x6pg288jp-gdk-pixbuf-2.42.10 8qm6sjqa09a03glzmafprpp69k74l4lm-binutils-2.40 8xk4yl1r3n6kbyn05qhan7nbag7npymx-glibc-2.35-224 8zg9l6rsr08ydy4bqsjawdc7gwyrjfnj-gtk+3-3.24.36 a46g6jg1rckzr7s965pb7kibr315k070-dconf-0.40.0-lib cp0a1s7dcnpbnpk0w010c3jw2dchybsv-gettext-0.21 dncrri4lnm9hg51vlxrfxfvrq7cbdjz0-libselinux-3.3 drzq6hwdckc1vz6vcjwcvigw5y00viki-pango-1.50.12 dvmcyy8mrr11i2ah2qhdn98mp86y2msa-cairo-1.16.0 f51ngj5s5qa21n4avcg8gkfsdakn4457-emacs-pgtk-20230412.0 fd9dpbx6kdvhg201cg7sb62mlcxr59g1-tree-sitter-0.20.8 ja73jc04spx5ycdfidx4v7f93vhmq3g0-libxml2-2.10.3 jlpjrfcqsdjyr3kic7qky533xndgf1cc-libpng-apng-1.6.39 kvj4j4ssr2abq4583yc31nvswdf9vf38-sqlite-3.41.1 l5axanhh6k6lkgfmap9ijlkm0fm306s3-gmp-with-cxx-6.2.1 n3q6s6bx8cah2afxz03rf4bk54dp6hwk-tree-sitter-grammars n8nrvcjddnic00c0ahpkhndjjfashwqf-jansson-2.14 nbyh8bpka52cmlcx44wrqlb4rkwivf1m-at-spi2-core-2.46.0 ngh5s155lcm1hwsakrfhla536x9g5v4z-binutils-wrapper-2.40 p975i9blgmkjfxpnlvdmm0xvjg573b6l-gcc-12.2.0 qgb2lkzch2knryn4dv7q5xpfpjj16rk7-giflib-5.2.1 rj3p4qccj2mkr95wdri7glh3j0xfjjhr-freetype-2.13.0 sb09vr7ycnl60phi4ifaqxr50nwn52y2-fontconfig-2.14.0-lib vn5s2ckdxjsmdxmm961w3rmg8ayhm6hh-gnutls-3.8.0 w48wlcgks7qd6ajcgvv5qwjihw4ambia-dbus-1.14.4-lib wx5j7zk8cn7spmyq1f15i6yymakvgp8p-libjpeg-turbo-2.1.4 xfkzh15vxmxhiza2s2din1az0vzfaf56-zlib-1.2.13
Deriver: 8spvih5s093qkxp96gcppafpx1hc62kl-emacs-pgtk-20230412.0.drv
Sig: nix-community.cachix.org-1:89BGhWbyWvF+vdR7r9Aa+DnaT33o/bPx7TBsG1gVGVxZA88+lSvbDlPE29yzQ+XQNPbd4gfjMsEIVp5/WKvKAw==

So I did input revision of emacs-overlay to my flake inputs (did not change nixpkgs) and got the binary cache.

Also someone said that github action does not populate cache, hydra is the one that builds for it.

For reference :

  1. My flake.nix

  2. Package path - /nix/store/2nbd2cc7gnbq516hjagmszbb2vzifmr4-emacs-pgtk-with-packages-20230412.0/bin/emacs

    You can see the lock file in the repo itself. Hopefully whenever you update you get the binary cache!

joshbode commented 1 year ago

Also someone said that github action does not populate cache, hydra is the one that builds for it.

Yes - I believe you're right - the Cachix action warning is a red-herring.

joshbode commented 1 year ago

I actually think it's possibly a timing issue.

It looks like the Hydra build built from:

(see the build inputs)

However, if I pin the emacs-overlay registry to that commit:

$ nix registry pin emacs-overlay github:nix-community/emacs-overlay/1be00f42d07320f4fd0230172ceb27ec40330f53
$ nix flake metadata emacs-overlay
Resolved URL:  github:nix-community/emacs-overlay/1be00f42d07320f4fd0230172ceb27ec40330f53
Locked URL:    github:nix-community/emacs-overlay/1be00f42d07320f4fd0230172ceb27ec40330f53
Description:   Bleeding edge Emacs overlay
Path:          /nix/store/6bq23wzpfipa28y77asm6q9dkgzmp870-source
Revision:      1be00f42d07320f4fd0230172ceb27ec40330f53
Last modified: 2023-04-12 13:40:40
Inputs:
├───flake-utils: github:numtide/flake-utils/5aed5285a952e0b949eb3ba02c12fa4fcfef535f
└───nixpkgs: github:NixOS/nixpkgs/0f213d0fee84280d8c3a97f7469b988d6fe5fcdf

we see that the flake depends upon github:NixOS/nixpkgs/0f213d0 (not github:NixOS/nixpkgs/db24d86 as was used in the build). Note NixOS/nixpkgs/0f213d0 is from 2023-01-13

joshbode commented 1 year ago

Possibly this line in hydra/spec.json needs to be changed to:

-      "value": "https://github.com/NixOS/nixpkgs.git nixos-unstable",
+      "value": "https://github.com/NixOS/nixpkgs.git 0f213d0",

to match the flake.lock as nixos-unstable is a moving target. I'm not sure if there's a better way to keep these synchronised...

luftmensch-luftmensch commented 1 year ago

Any updates on this topic? I'm getting the same result as above

$ nix flake metadata emacs-overlay
Resolved URL:  github:nix-community/emacs-overlay
Locked URL:    github:nix-community/emacs-overlay/8f327ba75dc95080ffab59a4f68b2c73b52d3cb3
Description:   Bleeding edge Emacs overlay
Path:          /nix/store/5fsfnmzr1icrm8lf01k8rkjdjnkid8vc-source
Revision:      8f327ba75dc95080ffab59a4f68b2c73b52d3cb3
Last modified: 2023-05-24 20:25:10
Inputs:
├───flake-utils: github:numtide/flake-utils/5aed5285a952e0b949eb3ba02c12fa4fcfef535f
└───nixpkgs: github:NixOS/nixpkgs/0f213d0fee84280d8c3a97f7469b988d6fe5fcdf
idlip commented 1 year ago

@luftmensch-luftmensch You mean, you are not getting the binary cache for emacs-overlay?

luftmensch-luftmensch commented 1 year ago

Yup, although even if I'm failing to build from source due to emacs-let-alist failing with:

Wrong type argument: package-desc, nil

idlip commented 1 year ago

Is it with Pgtk?

Cause I see emacsGit and emacsUnstable is available in cache. emacsPgtk is not available. I have asked about this in matrix before, no idea on this.

I had issue last time I updated, there was no Pgtk, but in the next hydra build I got pgtk cache.

You can very it here: https://hydra.nix-community.org/jobset/emacs-overlay/unstable

Goto Package > Details > Note the hash in output store path

curl https://nix-community.cachix.org/vv447g3l7jvnlqikbn86hlbcawxjcs36.narinfo ==> For Pgtk latest atm (not available)

curl https://nix-community.cachix.org/bj0faiikf8j14kbh90zl10l94nhbabsc.narinfo ==> For emacsUnstable (available)

luftmensch-luftmensch commented 1 year ago

@idlip Yes, I forgot to mention it was the Pgtk version (as I'm using it under wayland). Do you know if there is some reason for this absence? Or a work around solution?

leungbk commented 1 year ago

Yup, although even if I'm failing to build from source due to emacs-let-alist failing with:

Wrong type argument: package-desc, nil

This is almost certainly a native-comp issue: https://lists.gnu.org/archive/html/bug-gnu-emacs/2023-05/msg01517.html. It's not anything to do with Cachix.

adisbladis commented 1 year ago

With https://github.com/nix-community/emacs-overlay/pull/330 and https://github.com/nix-community/emacs-overlay/commit/908711c0c6eb1160ea0cbaf3187200674609d661 this should be less of an issue going forward.

joshbode commented 1 year ago

Great! #330 appears to have addressed this for me:

$ HASH=$(basename $(nix eval --raw emacs-overlay#emacs-pgtk) | cut -d- -f1)
$ curl https://nix-community.cachix.org/${HASH}.narinfo
StorePath: /nix/store/5czcc935qpnws9ddiinq71ri3r2c9zv5-emacs-pgtk-20230531.0
URL: nar/439e682c3661a65d467650b6b90b98e34894fa4ddcdcd09c533f06b806b3861b.nar.zst
Compression: zstd
FileHash: sha256:439e682c3661a65d467650b6b90b98e34894fa4ddcdcd09c533f06b806b3861b
FileSize: 102599633
NarHash: sha256:04br9026y1dr820k9s7vxly3rld34mahdjsa7mawbinnpfqm6dcj
NarSize: 325404056
References: ...
Deriver: z3n4bv3pdhda296bv0q2ld3dnrcjp581-emacs-pgtk-20230531.0.drv
Sig: nix-community.cachix.org-1:l+H6gkFUUXU+DmAHwPm9GPYjOi/gL/qiGIhV/TEq/WbF2YVNowEk3geQe3rZTrwLHte85CYoUIvO/lcGrwudAg==

When I run:

$ nix profile install emacs-overlay#emacs-pgtk

it is pulling a recent build from Cachix :)

joshbode commented 1 year ago

thanks @adisbladis