Can we realize better nix store cache in home-manager and the container CI/CD?

kachick commented 5 months ago

I tried to use GitHub action cache for it ever, but less or no benefit exist in https://github.com/kachick/asdf2json/blob/1d47e12dc13689ffb7444111f075f4df9e70790b/.github/workflows/ci.yml. And reverted

https://github.com/NixOS/nix/pull/9474#issuecomment-1831899278 👀 https://github.com/NixOS/nix/issues/3183#issuecomment-548367347 👀

kachick commented 5 months ago

https://www.haskellforall.com/2022/10/how-to-correctly-cache-build-time.html 👁️

kachick commented 5 months ago

At least, podman cp is slow and can't complete even in my local...

dotfiles main(12a48df)  ≡via 🐹 v1.22.1 via ❄️  impure (nix-shell-env) zsh
125! podman cp b8f6f48e342b75c53f29776a62e362ed1521a58426dfee02d96b7a68a21c2ce2:/home/user/.profile ./.profile.copied

dotfiles main(12a48df) * ≡via 🐹 v1.22.1 via ❄️  impure (nix-shell-env) zsh
> bat ./.profile.copied
───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: ./.profile.copied
───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ . "/home/user/.nix-profile/etc/profile.d/hm-session-vars.sh"
   2   │
   3   │ export INPUTRC="/home/user/.inputrc"
   4   │
   5   │ # Used same method as switching to fish
   6   │ # https://wiki.archlinux.org/title/fish#Setting_fish_as_interactive_shell_only
   7   │ if [[ $(/nix/store/g1v19g0a6zs4np9ac9vxkb1mf7x1qcw9-procps-3.3.17/bin/ps --no-header --pid=$PPID --format=comm) != "zsh" && -z ${BASH_EXECUTION_STRING} ]]
   8   │ then
   9   │   shopt -q login_shell && LOGIN_OPTION='--login' || LOGIN_OPTION=""
  10   │   exec /nix/store/w81n67bahxz0ll5avk5kdj25ridwf6y8-zsh-5.9/bin/zsh $LOGIN_OPTION
  11   │ fi
  12   │
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

dotfiles main(12a48df) * ≡via 🐹 v1.22.1 via ❄️  impure (nix-shell-env) zsh
> podman cp b8f6f48e342b75c53f29776a62e362ed1521a58426dfee02d96b7a68a21c2ce2:/nix/store ./nix-store-cache.copied
Error: 2 errors occurred:
    * copying to host: copier: put: error creating "/Users/kachick/repos/dotfiles/nix-store-cache.copied/032wiarm65zp3bh9ak3dz2sqcr3n8g70-bash-interactive-5.2p26/bin": mkdir /Users/kachick/repos/dotfiles/nix-store-cache.copied/032wiarm65zp3bh9ak3dz2sqcr3n8g70-bash-interactive-5.2p26/bin: permission denied
    * copying from container: io: read/write on closed pipe

dotfiles main(12a48df) * ≡via 🐹 v1.22.1 2m29s via ❄️  impure (nix-shell-env) zsh
125! la ./nix-store-cache.copied
drwxr-xr-x    - kachick 12 3  23:37 .links
dr-xr-xr-x    - kachick  1 1   1970 032wiarm65zp3bh9ak3dz2sqcr3n8g70-bash-interactive-5.2p26
.r--r--r-- 2.9k kachick  1 1   1970 00hkkgnzq591sh6vb7z6v5hgngp7pvps-libselinux-3.6.tar.gz.drv
.r--r--r--  437 kachick  1 1   1970 00qr10y7z2fcvrp9b2m46710nkjvj55z-update-autotools-gnu-config-scripts.sh
.r--r--r-- 4.4k kachick  1 1   1970 00zn8nb0f5x19ckihvnk7y69l7vx2gf5-fix-implicit-getdef_bool.patch.drv
.r--r--r-- 1.5k kachick  1 1   1970 001gp43bjqzx60cg345n2slzg7131za8-nix-nss-open-files.patch
.r--r--r-- 1.8k kachick  1 1   1970 007bzydbwwn5jz86dvqb70755hvz4nq4-wait-timeout-0.2.0.drv
.r--r--r-- 2.9k kachick  1 1   1970 01fc3rmg7vnpkpsqiqrpd50pg84idfnw-Test-NoWarnings-1.06.tar.gz.drv
.r--r--r-- 2.9k kachick  1 1   1970 01jkwncca9gkjv0vzhzaa8jw15wc4964-openssl-3.0.13.tar.gz.drv
.r--r--r-- 8.8k kachick  1 1   1970 01nyy2xyv69mlfr16fz80gyn5fd7nlvw-microlens-0.4.13.1.drv
.r--r--r-- 2.9k kachick  1 1   1970 01390vsvxf4a82bnl9aa8awws7p42692-code-page-0.2.1.tar.gz.drv
.r--r--r-- 8.8k kachick  1 1   1970 02i6ky14s20bjv2nzi6ii9cwwqnnllc4-easy-file-0.2.5.drv
.r--r--r-- 2.9k kachick  1 1   1970 02v18qqbm1lc95ki7cd49ndlf5mvnmy5-yasm-1.3.0.tar.gz.drv
.r--r--r-- 3.4k kachick  1 1   1970 026bhm689m0syvgl9srwxd54dwkzkif9-source.drv

phanirithvij commented 2 months ago

Hi I suggest https://github.com/DeterminateSystems/magic-nix-cache-action/ which uses magic-nix-cache (https://github.com/DeterminateSystems/magic-nix-cache) which is an attic server backed by github actions cache. Things of note:

10GB limit per repository limit on github cache (github has this limit)
After reaching that limit lru cache eviction happens (older cache entries will be deleted by github)
If doing too many nix store operations you will hit 429 (rate limits) but once the workflows run enough times it will all be cached to gha cache.
all of these are mentioned in that repo's readme but thought I would share since I recently started using it.

I don't know if I personally will use cachix service because it is proprietary but they do have other useful oss tools that I will use.

Edit: just saw https://github.com/kachick/dotfiles/pull/548/files#diff-20bb9d539cb32024b86ca3ae4683e92a8463f48df623db5a7d54803d9d81cb1bL44 you already are using it, sorry I missed it. Ignore my comment. I am closing it as duplicate.

kachick commented 2 months ago

Thanks for your suggestion! Yes, that action was helpful if the repository has a long build time. However, in the case of this repository, the build time per package is fast. Nonetheless, using many dependencies causes rate-limit problems rather than providing benefits... 😢

kachick / dotfiles

Can we realize better nix store cache in home-manager and the container CI/CD? #472