search

Mic92 / nixpkgs-review

Review pull-requests on https://github.com/NixOS/nixpkgs
MIT License
353 stars 59 forks source link

error: cannot find flake 'flake:None' in the flake registries #354

Closed gepbird closed 8 months ago

gepbird commented 10 months ago

I usually see this kind of error about 1 out of 5 pull request reviews. The common thing I noticed in them is some packages are marked as broken and skipped and at the end I get the 'flake:None' error.

Once I pressed Ctrl+Z to pause the review, and after ~10 minutes I run fg to resume it. This caused some curl requests to time out. Another event that might be relevant is I ran out of memory during a big rebuild, so that might have messed up my nix store? I'm not sure. After force restarting due to the out of memory incident, I wanted to review another package that is very small and just a 1 rebuilt. I run nixpkgs-review, I see git stuff printed out, fetching the remote was done, but it didn't get to the nix build part, because it was using ~7G of memory and I got OOM again. After another force restart build the small package built pretty quickly and didn't use nearly as much memory.

I successfully reproduced this bug every time on the same machine (with the PRs that have already failed due to thib bug). If I recall correctly it happened before any pausing or OOM incidents. I tried re-cloning the nixpkgs repo. I tried cleaning the nix store with nix-collect-garbage --older-than 30d to remove packages generated by reviewing PRs. I tried restarting my machine after a review then run the same review again but nothing worked. Keep in mind I tried all of these potential fixes after having the unpause and OOM incidents and I still got this error.

This is not a problem with the package, since other people can build it.

Here's the full log of a PR review:

nixos.nixpkgs ❯ nixpkgs-review pr https://github.com/NixOS/nixpkgs/pull/251536
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/251536/head:refs/nixpkgs-review/1
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (4/4), done.
remote: Total 7 (delta 4), reused 4 (delta 4), pack-reused 3
Unpacking objects: 100% (7/7), 4.83 KiB | 449.00 KiB/s, done.
From https://github.com/NixOS/nixpkgs
 + ee5330729205...4bbc004a216e refs/pull/251536/head -> refs/nixpkgs-review/1  (forced update)
$ git worktree add /home/gep/.cache/nixpkgs-review/pr-251536-8/nixpkgs c9c99c053abe3c6be42d3f4d7deedd7282103567
Preparing worktree (detached HEAD c9c99c053abe)
Updating files: 100% (36898/36898), done.
HEAD is now at c9c99c053abe Merge pull request #251844 from fabaff/python-engineio-bump
$ git merge --no-commit --no-ff 4bbc004a216e5e24730b2c59fc5cb73456e1ac77
Automatic merge went well; stopped before committing as requested
$ nix build --extra-experimental-features nix-command no-url-literals --no-link --keep-going --no-allow-import-from-derivation --option build-use-sandbox relaxed -f /home/gep/.cache/nixpkgs-review/pr-251536-8/build.nix
error: builder for '/nix/store/sbllc7k3r8l5d576fg0vxfv5b9kjbsqf-vmware-modules-17.0.2-6.4.11.drv' failed with exit code 2;
       last 10 log lines:
       >  1413 |    segs = skb_gso_segment(skb, 0);
       >       |         ^
       >   CC [M]  /build/source/vmnet-only/smac.o
       > cc1: some warnings being treated as errors
       > make[3]: *** [/nix/store/80pzdqwj98vb97ls4rnfljshkjsjdip2-linux-6.4.11-dev/lib/modules/6.4.11-hardened1/source/scripts/Makefile.build:252: /build/source/vmnet-only/bridge.o] Error 1
       > make[3]: *** Waiting for unfinished jobs....
       > make[2]: *** [/nix/store/80pzdqwj98vb97ls4rnfljshkjsjdip2-linux-6.4.11-dev/lib/modules/6.4.11-hardened1/source/Makefile:2032: /build/source/vmnet-only] Error 2
       > make[1]: *** [Makefile:117: vmnet.ko] Error 2
       > make[1]: Leaving directory '/build/source/vmnet-only'
       > make: *** [Makefile:21: vmnet-only] Error 2
       For full logs, run 'nix log /nix/store/sbllc7k3r8l5d576fg0vxfv5b9kjbsqf-vmware-modules-17.0.2-6.4.11.drv'.
error: builder for '/nix/store/0h8m39bw5pgw5py84wrzvkczm4xr3yl2-vmware-modules-17.0.2-6.4.11.drv' failed with exit code 2;
       last 10 log lines:
       >   MODULEBUILDDIR= postbuild
       > make[2]: Entering directory '/build/source/vmmon-only'
       > make[2]: 'postbuild' is up to date.
       > make[2]: Leaving directory '/build/source/vmmon-only'
       > cp -f vmmon.ko ./../vmmon.o
       > make[1]: Leaving directory '/build/source/vmmon-only'
       > make[2]: *** [/nix/store/wmskc9g3zhifd35k9qzhazq66v0cf9lf-linux-6.4.11-dev/lib/modules/6.4.11-xanmod1/source/Makefile:2187: /build/source/vmnet-only] Error 2
       > make[1]: *** [Makefile:117: vmnet.ko] Error 2
       > make[1]: Leaving directory '/build/source/vmnet-only'
       > make: *** [Makefile:21: vmnet-only] Error 2
       For full logs, run 'nix log /nix/store/0h8m39bw5pgw5py84wrzvkczm4xr3yl2-vmware-modules-17.0.2-6.4.11.drv'.
error: builder for '/nix/store/83x9qf8372ndzkmj3wkz4nb15iqyaz9f-vmware-modules-17.0.2-6.4.12.drv' failed with exit code 2;
       last 10 log lines:
       >   LD [M]  /build/source/vmmon-only/vmmon.ko
       >   BTF [M] /build/source/vmmon-only/vmmon.ko
       > Skipping BTF generation for /build/source/vmmon-only/vmmon.ko due to unavailability of vmlinux
       > make -C $PWD SRCROOT=$PWD/. \
       >   MODULEBUILDDIR= postbuild
       > make[2]: Entering directory '/build/source/vmmon-only'
       > make[2]: 'postbuild' is up to date.
       > make[2]: Leaving directory '/build/source/vmmon-only'
       > cp -f vmmon.ko ./../vmmon.o
       > make[1]: Leaving directory '/build/source/vmmon-only'
       For full logs, run 'nix log /nix/store/83x9qf8372ndzkmj3wkz4nb15iqyaz9f-vmware-modules-17.0.2-6.4.12.drv'.
error: builder for '/nix/store/i0scq9i7yfxz6cfw0j677f4zd492swcj-vmware-modules-17.0.2-6.4.12.drv' failed with exit code 2;
       last 10 log lines:
       >   LD [M]  /build/source/vmmon-only/vmmon.ko
       >   BTF [M] /build/source/vmmon-only/vmmon.ko
       > Skipping BTF generation for /build/source/vmmon-only/vmmon.ko due to unavailability of vmlinux
       > make -C $PWD SRCROOT=$PWD/. \
       >   MODULEBUILDDIR= postbuild
       > make[2]: Entering directory '/build/source/vmmon-only'
       > make[2]: 'postbuild' is up to date.
       > make[2]: Leaving directory '/build/source/vmmon-only'
       > cp -f vmmon.ko ./../vmmon.o
       > make[1]: Leaving directory '/build/source/vmmon-only'
       For full logs, run 'nix log /nix/store/i0scq9i7yfxz6cfw0j677f4zd492swcj-vmware-modules-17.0.2-6.4.12.drv'.
error: builder for '/nix/store/njw8d2acnciigaamvqnsd3dl37v64dmg-vmware-modules-17.0.2-6.4.12.drv' failed with exit code 2;
       last 10 log lines:
       >   LD [M]  /build/source/vmmon-only/vmmon.ko
       >   BTF [M] /build/source/vmmon-only/vmmon.ko
       > Skipping BTF generation for /build/source/vmmon-only/vmmon.ko due to unavailability of vmlinux
       > make -C $PWD SRCROOT=$PWD/. \
       >   MODULEBUILDDIR= postbuild
       > make[2]: Entering directory '/build/source/vmmon-only'
       > make[2]: 'postbuild' is up to date.
       > make[2]: Leaving directory '/build/source/vmmon-only'
       > cp -f vmmon.ko ./../vmmon.o
       > make[1]: Leaving directory '/build/source/vmmon-only'
       For full logs, run 'nix log /nix/store/njw8d2acnciigaamvqnsd3dl37v64dmg-vmware-modules-17.0.2-6.4.12.drv'.
error: 5 dependencies of derivation '/nix/store/f9pzbmgd0f4gvq0v58wk932fj9wicb3r-review-shell.drv' failed to build

Link to currently reviewing PR:
https://github.com/NixOS/nixpkgs/pull/251536

8 packages marked as broken and skipped:
linuxKernel.packages.hardkernel_4_14.vmware linuxKernel.packages.linux_4_14_hardened.vmware linuxKernel.packages.linux_4_19_hardened.vmware linuxKernel.packages.linux_5_4_hardened.vmware linuxPackages_4_14_hardened.vmware linuxPackages_4_19_hardened.vmware linuxPackages_5_4_hardened.vmware linuxPackages_hardkernel_latest.vmware

5 packages failed to build:
linuxPackages_latest.vmware linuxPackages_6_4_hardened.vmware linuxPackages_latest-libre.vmware linuxPackages_testing_bcachefs.vmware linuxPackages_xanmod_latest.vmware

14 packages built:
linuxKernel.packages.linux_4_14.vmware linuxKernel.packages.linux_4_19.vmware linuxKernel.packages.linux_5_10.vmware linuxPackages_5_10_hardened.vmware linuxKernel.packages.linux_5_15.vmware linuxPackages_5_15_hardened.vmware linuxKernel.packages.linux_5_4.vmware linuxPackages.vmware linuxPackages_hardened.vmware linuxPackages-libre.vmware linuxPackages_lqx.vmware linuxPackages_xanmod.vmware linuxPackages_zen.vmware vmware-workstation

error: cannot find flake 'flake:None' in the flake registries
error: cannot find flake 'flake:None' in the flake registries
error: cannot find flake 'flake:None' in the flake registries
error: cannot find flake 'flake:None' in the flake registries
error: cannot find flake 'flake:None' in the flake registries
error: cannot find flake 'flake:None' in the flake registries
error: cannot find flake 'flake:None' in the flake registries
error: cannot find flake 'flake:None' in the flake registries
$ /nix/store/xlzipy8vsj28b1fg8vmlwhq585zlaqj8-nix-2.17.0/bin/nix-shell /home/gep/.cache/nixpkgs-review/pr-251536-8/shell.nix

[nix-shell:~/.cache/nixpkgs-review/pr-251536-8]$
gepbird commented 10 months ago

The extremely long build time that seems like it never finishes can be fixed by clearing nixpkgs-review caches.

Before clearing caches I waited 20 minutes, nix-build command allocated 6G of memory and I had to interrupt it, here's a timelapse as proof:

https://github.com/Mic92/nixpkgs-review/assets/29818440/5143c02f-72db-496a-b20f-2f3986d60ed5

After clearing caches it was done under a minute, only allocating 200-300M of memory:

https://github.com/Mic92/nixpkgs-review/assets/29818440/c215fa35-9119-404f-b0e2-5e73b8db56d5

However this doesn't solve my issue, the error: cannot find flake 'flake:None' in the flake registries issue still persists.

figsoda commented 10 months ago

This is a bit confusing because I don't think we use flakes at all, but None also makes it almost certain that it's nixpkgs-review and not nix or nixpkgs

Mic92 commented 10 months ago

Can you post the content of build.nix in those cases?

gepbird commented 10 months ago

Can you post the content of build.nix in those cases?

@Mic92 Of course! Here you go: build.nix.txt

Github didn't let me upload it with a nix extension so I added .txt.

Artturin commented 8 months ago

Debugging with

python -m trace --ignore-dir "$(python -c 'import os, sys; print(os.pathsep.join(sys.path[1:]))')" -t $NIXGITS/nixpkgs-review/result/bin/.nixpkgs-review-wrapped pr 261631

from a comment here https://stackoverflow.com/a/33449763

led to this https://github.com/Mic92/nixpkgs-review/blob/1a9131b8686dc7a15980a2aed0733bf19a5a9313/nixpkgs_review/report.py#L74

added print(f"HERE {path}") above the nix_log

HERE /nix/store/58mj37hgqg7lm5gixjba3x50285idxcx-brise-unstable-2017-09-16.drv^*
HERE /nix/store/1cfdr8p8fq97gmddp12xr1jfnrph9p0l-emacs-rime-20230212.1425.drv^*
HERE /nix/store/jkbf33fimig0qba2xbgqh5k298ggiqf2-fcitx5-chinese-addons-5.1.1.drv^*
HERE /nix/store/pwhwzxyyh6frf5wby3isx53abswrw9fb-fcitx5-rime-5.1.2.drv^*
HERE /nix/store/mz9gynnnax52pyslkxs4m94c11cw1889-goldendict-ng-23.09.10.drv^*
HERE /nix/store/2d93awdkb7qm9ixfkzlkn9v3x7cbrc0c-ibus-libpinyin-1.15.3.drv^*
HERE /nix/store/c15ygkw94z098qhzba2rki5hjq93m266-ibus-rime-1.5.0.drv^*
HERE /nix/store/wdxilqmkgqjhidj21cijf3rj4kdyk1sn-librime-1.9.0.drv^*
HERE /nix/store/mf1rhlszvpij47llh24wkvnvhpi6lk2z-opencc-1.1.7.drv^*
HERE /nix/store/4ad82nrj0n88xabzp3zjw233g73bdgja-rime-data-0.38.20211002.drv^*
HERE None^*
error: cannot find flake 'flake:None' in the flake registries

build.nix

{ pkgs ? import ./nixpkgs { system = "x86_64-linux"; config = import /tmp/tmpu07h9sh7.nix; } }:
with pkgs;
let
  paths = [
    brise
    emacsPackages."rime"
    fcitx5-chinese-addons
    fcitx5-rime
    goldendict-ng
    ibus-engines."libpinyin"
    ibus-engines."rime"
    librime
    opencc
    rime-data
  ];
  env = buildEnv {
    name = "env";
    inherit paths;
    ignoreCollisions = true;
  };
in (import ./nixpkgs { }).mkShell {
  name = "review-shell";
  preferLocalBuild = true;
  allowSubstitutes = false;
  dontWrapQtApps = true;
  packages = if builtins.length paths > 50 then [ env ] else paths;
}

Added a print(f"HERE {attr}") and

HERE None^*
HERE Attr(name='goldendict', exists=True, broken=True, blacklisted=False, path=None, drv_path=None, aliases=[], _path_verified=None)
error: cannot find flake 'flake:None' in the flake registries
Artturin commented 8 months ago

https://github.com/Mic92/nixpkgs-review/pull/369