Closed teto closed 2 years ago
Huh, the two assert(newInfo.ca)
in derivation-goal.cc
look good to me. Are you repairing or checking paths by any chance?
I am just running a nixos-rebuild, at some point it's possible it calls builtins.pathExists.
something I've noticed is that happens after I rebase on nixos-unstable. It rebuilds something and fails once, then I restart the command and it works fine until the next rebase.
Hmm, that does sound like something to do with fixed output derivations.
Just came across the same error while running nixops
, also appears to be nondeterministic. I'm way out of my depth here but thought I'd post some of my info in case it's to any use.
> nix --version
nix (Nix) 3.0pre20201020_e0ca98c
> sudo coredumpctl debug
PID: 15627 (nix-daemon)
UID: 0 (root)
GID: 0 (root)
Signal: 6 (ABRT)
Timestamp: Thu 2020-11-26 12:16:02 CET (2h 3min ago)
Command Line: nix-daemon 15622
Executable: /nix/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/bin/nix
Control Group: /system.slice/nix-daemon.service
Unit: nix-daemon.service
Slice: system.slice
Boot ID: 9fd49a62ddd1443dbee27f2b36fc192a
Machine ID: b2a66e78323a41eea3abac32cebf1bef
Hostname: nixpix
Storage: /var/lib/systemd/coredump/core.nix-daemon.0.9fd49a62ddd1443dbee27f2b36fc192a.15627.1606389362000000.lz4
Message: Process 15627 (nix-daemon) of user 0 dumped core.
The stacktrace from gdb:
(gdb) thread apply all bt
Thread 3 (Thread 0x7fc97200c700 (LWP 21283)):
#0 0x00007fc974292dd9 in poll () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6
#1 0x00007fc973d31596 in Curl_poll () from /nix/store/wmp5y493sfx864nrmn4gy94gza0d30ck-curl-7.72.0/lib/libcurl.so.4
#2 0x00007fc973d26ac3 in Curl_multi_wait.part.0 () from /nix/store/wmp5y493sfx864nrmn4gy94gza0d30ck-curl-7.72.0/lib/libcurl.so.4
#3 0x00007fc973d26ca3 in curl_multi_wait () from /nix/store/wmp5y493sfx864nrmn4gy94gza0d30ck-curl-7.72.0/lib/libcurl.so.4
#4 0x00007fc9749968d1 in nix::curlFileTransfer::workerThreadMain() () from /nix/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixstore.so
#5 0x00007fc974997e0c in nix::curlFileTransfer::workerThreadEntry() () from /nix/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixstore.so
#6 0x00007fc9745b9930 in ?? () from /nix/store/vran8acwir59772hj4vscr7zribvp7l5-gcc-9.3.0-lib/lib/libstdc++.so.6
#7 0x00007fc97436bead in start_thread () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libpthread.so.0
#8 0x00007fc97429cd2f in clone () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6
Thread 2 (Thread 0x7fc97180b700 (LWP 15628)):
#0 0x00007fc974292dd9 in poll () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6
#1 0x00000000004ed6de in std::thread::_State_impl<std::thread::_Invoker<std::tuple<nix::MonitorFdHup::MonitorFdHup(int)::{lambda()#1}> > >::_M_run() ()
#2 0x00007fc9745b9930 in ?? () from /nix/store/vran8acwir59772hj4vscr7zribvp7l5-gcc-9.3.0-lib/lib/libstdc++.so.6
#3 0x00007fc97436bead in start_thread () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libpthread.so.0
#4 0x00007fc97429cd2f in clone () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6
Thread 1 (Thread 0x7fc9731c9a00 (LWP 15627)):
#0 0x00007fc9741dd08a in raise () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6
#1 0x00007fc9741c7528 in abort () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6
#2 0x00007fc9741c740f in __assert_fail_base.cold.0 () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6
#3 0x00007fc9741d5a02 in __assert_fail () from /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6
#4 0x00007fc97490d5cc in nix::DerivationGoal::registerOutputs() () from /nix/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixstore.so
#5 0x00007fc97490e944 in nix::DerivationGoal::buildDone() () from /nix/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixstore.so
#6 0x00007fc97495a603 in nix::Worker::run(std::set<std::shared_ptr<nix::Goal>, nix::CompareGoalPtrs, std::allocator<std::shared_ptr<nix::Goal> > > const&) () from /nix/store/d3mli5a7ac4li
9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixstore.so
#7 0x00007fc97494df5b in nix::LocalStore::buildPaths(std::vector<nix::StorePathWithOutputs, std::allocator<nix::StorePathWithOutputs> > const&, nix::BuildMode) () from /nix/store/d3mli5a7
ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixstore.so
#8 0x00007fc97496d1d9 in nix::daemon::processConnection(nix::ref<nix::Store>, nix::FdSource&, nix::FdSink&, nix::daemon::TrustedFlag, nix::daemon::RecursiveFlag, std::function<void (nix::
Store&)>) () from /nix/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixstore.so
#9 0x00000000004b4215 in std::_Function_handler<void (), daemonLoop(char**)::{lambda()#1}>::_M_invoke(std::_Any_data const&) ()
#10 0x00007fc974797cac in std::_Function_handler<void (), nix::startProcess(std::function<void ()>, nix::ProcessOptions const&)::{lambda()#1}>::_M_invoke(std::_Any_data const&) () from /ni
x/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixutil.so
#11 0x00007fc974794659 in nix::doFork(bool, std::function<void ()>) () from /nix/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixutil.so
#12 0x00007fc974797b99 in nix::startProcess(std::function<void ()>, nix::ProcessOptions const&) () from /nix/store/d3mli5a7ac4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixuti
l.so
#13 0x00000000004b53d3 in main_nix_daemon(int, char**) ()
#14 0x000000000056b29b in nix::mainWrapped(int, char**) ()
#15 0x00007fc974bf3ddd in nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) () from /nix/store/d3mli5a7a
c4li9ixxnjxyn2dhyv1v7ws-nix-3.0pre20201020_e0ca98c/lib/libnixmain.so
#16 0x00000000004702eb in main ()
Last output from nixops:
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/hzgrsyvqdm7bidisfxy8z9n0jx32a1bb-c-ares-1.15.0
shrinking /nix/store/hzgrsyvqdm7bidisfxy8z9n0jx32a1bb-c-ares-1.15.0/lib/libcares.so.2.3.0
CC networking/udhcp/common.o
gzipping man pages under /nix/store/hzgrsyvqdm7bidisfxy8z9n0jx32a1bb-c-ares-1.15.0/share/man/
CC networking/udhcp/d6_dhcpc.o
strip is /nix/store/6k3b0kpajqnzir2603vpyzi7qqhs1l31-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/hzgrsyvqdm7bidisfxy8z9n0jx32a1bb-c-ares-1.15.0/lib
patching script interpreter paths in /nix/store/hzgrsyvqdm7bidisfxy8z9n0jx32a1bb-c-ares-1.15.0
mv -f .deps/dirsearch.Tpo .deps/dirsearch.Po
g++ -DHAVE_CONFIG_H -I. -I. -I./../include -I./../elfcpp -DLOCALEDIR="\"/nix/store/r52p03fm2nd3xsjvlp7nrqp5a23l6qxk-armv7l-unknown-linux-musleabihf-binutils-2.31.1/share/locale\"" -DBINDI
R="\"/nix/store/r52p03fm2nd3xsjvlp7nrqp5a23l6qxk-armv7l-unknown-linux-musleabihf-binutils-2.31.1/bin\"" -DTOOLBINDIR="\"/nix/store/r52p03fm2nd3xsjvlp7nrqp5a23l6qxk-armv7l-unknown-linux-mus
leabihf-binutils-2.31.1/armv7l-unknown-linux-musleabihf/bin\"" -DTOOLLIBDIR="\"/nix/store/r52p03fm2nd3xsjvlp7nrqp5a23l6qxk-armv7l-unknown-linux-musleabihf-binutils-2.31.1/armv7l-unknown-li
nux-musleabihf/lib\"" -W -Wall -Wstack-usage=262144 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=dynobj.o -pthread -g -O2 -MT dynobj.o -MD -MP -MF .deps/dynobj.Tpo -c -o
dynobj.o dynobj.cc
checking for references to /build/ in /nix/store/hzgrsyvqdm7bidisfxy8z9n0jx32a1bb-c-ares-1.15.0...
error: --- EndOfFile ------------------------------------------------------------- nix-build
unexpected end-of-file
Traceback (most recent call last):
File "/nix/store/nfq1hckbwhcmw1b1c6spwsppp1px0pc4-nixops-1.7/bin/..nixops-wrapped-wrapped", line 991, in <module>
args.op()
File "/nix/store/nfq1hckbwhcmw1b1c6spwsppp1px0pc4-nixops-1.7/bin/..nixops-wrapped-wrapped", line 412, in op_deploy
max_concurrent_activate=args.max_concurrent_activate)
File "/nix/store/nfq1hckbwhcmw1b1c6spwsppp1px0pc4-nixops-1.7/lib/python2.7/site-packages/nixops/deployment.py", line 1063, in deploy
self.run_with_notify('deploy', lambda: self._deploy(**kwargs))
File "/nix/store/nfq1hckbwhcmw1b1c6spwsppp1px0pc4-nixops-1.7/lib/python2.7/site-packages/nixops/deployment.py", line 1052, in run_with_notify
f()
File "/nix/store/nfq1hckbwhcmw1b1c6spwsppp1px0pc4-nixops-1.7/lib/python2.7/site-packages/nixops/deployment.py", line 1063, in <lambda>
self.run_with_notify('deploy', lambda: self._deploy(**kwargs))
File "/nix/store/nfq1hckbwhcmw1b1c6spwsppp1px0pc4-nixops-1.7/lib/python2.7/site-packages/nixops/deployment.py", line 1003, in _deploy
self.configs_path = self.build_configs(dry_run=dry_run, repair=repair, include=include, exclude=exclude)
File "/nix/store/nfq1hckbwhcmw1b1c6spwsppp1px0pc4-nixops-1.7/lib/python2.7/site-packages/nixops/deployment.py", line 671, in build_configs
raise Exception("unable to build all machine configurations")
Exception: unable to build all machine configurations
The active derivation builder.sh
seems to have continued to completion after the fact.
Just got the same when rebuilding with flakes:
I've also been observing this on a bunch of systems. The weird thing is, restarting the same build a second time seems to just work?
I have just reproduced this with a Nix built with debug flags, so I now have a full backtrace with line and stack information and a 1.3 GB core file. Please let me know if you want the file to investigate this further then I can see if I can upload this somewhere.
Nix was built from 5a6ddb3de14a1684af6c793d663764d093fa7846 (current version of nixFlakes
in 20.09.20210206.002c001).
Describe the bug
nix crashes when rebuilding my flake based configuration. This is the second time this specific errors to me but it is not deterministic as rerunning the same command succeeds afterwards.
some context
and the coredump
Steps To Reproduce Too many custom repos involved. I hope the backtrace is enough.
Expected behavior