search

nix-community / nixos-anywhere

install nixos everywhere via ssh [maintainer=@numtide]
https://nix-community.github.io/nixos-anywhere/
MIT License
1.64k stars 116 forks source link

nixos-anywhere just kinda stops. #368

Closed dereulenspiegel closed 2 months ago

dereulenspiegel commented 2 months ago

Hello,

I am currently trying to migrate my homelab to nixos and for baremetal deployment I would like to use nixos-anywhere. As a first test I tried setting up a VM running under UTM in macOS (x86_64). Unfortunately nixos-anywhere just kinda seems to stop at some point and doesn't do anything. No partitioning is done on the target machine, no error (I picked up on) is emitted. Here is the debug log of my latest attempt after I let everything run for about 8 hours: nix run github:nix-community/nixos-anywhere -- --flake .#limiting-factor -L --debug root@192.168.205.5

+ shift
+ [[ 1 -gt 0 ]]
+ case "$1" in
+ [[ -z '' ]]
+ ssh_connection=root@192.168.205.5
+ shift
+ [[ 0 -gt 0 ]]
+ [[ y == \y ]]
+ nix_options+=("-L")
+ [[ y == \y ]]
+ nix_copy_options+=("--substitute-on-destination")
+ [[ -z '' ]]
+ [[ -z root@192.168.205.5 ]]
++ mktemp -d
+ ssh_key_dir=/tmp/tmp.VkXuGw8ixd
+ trap 'rm -rf "$ssh_key_dir"' EXIT
+ mkdir -p /tmp/tmp.VkXuGw8ixd
+ mkdir -p /Users/till/.ssh/
+ ssh-keygen -t ed25519 -f /tmp/tmp.VkXuGw8ixd/nixos-anywhere -P '' -C nixos-anywhere
+ [[ -n .#limiting-factor ]]
+ [[ .#limiting-factor =~ ^(.*)#([^#"]*)$ ]]
+ flake=.
+ flakeAttr=limiting-factor
+ [[ -z limiting-factor ]]
+ [[ n == \n ]]
+ [[ -n '' ]]
++ nix_build '.#nixosConfigurations."limiting-factor".config.system.build.diskoScript'
++ NIX_SSHOPTS='-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /tmp/tmp.VkXuGw8ixd/nixos-anywhere '
++ nix build --print-out-paths --no-link --extra-experimental-features 'nix-command flakes' --no-write-lock-file -L '.#nixosConfigurations."limiting-factor".config.system.build.diskoScript'
+ disko_script=/nix/store/hhdz1qxvr1dbyfy99i8c0qx6a1brvq91-disko
++ nix_build '.#nixosConfigurations."limiting-factor".config.system.build.toplevel'
++ NIX_SSHOPTS='-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /tmp/tmp.VkXuGw8ixd/nixos-anywhere '
++ nix build --print-out-paths --no-link --extra-experimental-features 'nix-command flakes' --no-write-lock-file -L '.#nixosConfigurations."limiting-factor".config.system.build.toplevel'
system-path> created 13069 symlinks in user environment
stage> checking syntax
+ nixos_system=/nix/store/rlmn9far0r9rf3nkam393qncy07z7p8r-nixos-system-nixos-24.05.20240903.6f6c45b
+ [[ -n '' ]]
+ [[ -n '' ]]
++ ssh -G root@192.168.205.5
+ ssh_settings='host 192.168.205.5
user root
hostname 192.168.205.5
port 22
addressfamily any
batchmode no
canonicalizefallbacklocal yes
canonicalizehostname false
checkhostip no
compression no
controlmaster true
enablesshkeysign no
clearallforwardings no
exitonforwardfailure no
fingerprinthash SHA256
forwardx11 no
forwardx11trusted no
gatewayports no
gssapiauthentication no
gssapidelegatecredentials no
hashknownhosts no
hostbasedauthentication no
identitiesonly no
kbdinteractiveauthentication yes
nohostauthenticationforlocalhost no
passwordauthentication yes
permitlocalcommand no
proxyusefdpass no
pubkeyauthentication true
requesttty auto
sessiontype default
stdinnull no
forkafterauthentication no
streamlocalbindunlink no
stricthostkeychecking ask
tcpkeepalive yes
tunnel false
verifyhostkeydns false
visualhostkey no
updatehostkeys true
enableescapecommandline no
canonicalizemaxdots 1
connectionattempts 1
forwardx11timeout 1200
numberofpasswordprompts 3
serveralivecountmax 2
serveraliveinterval 30
requiredrsasize 1024
obscurekeystroketiming yes
ciphers chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
controlpath /Users/till/.ssh/master-root@192.168.205.5:22
hostkeyalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
hostbasedacceptedalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
kexalgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
casignaturealgorithms ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
loglevel INFO
macs umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
securitykeyprovider internal
pubkeyacceptedalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
xauthlocation /usr/X11R6/bin/xauth
identityfile ~/.ssh/id_rsa
identityfile ~/.ssh/id_ecdsa
identityfile ~/.ssh/id_ecdsa_sk
identityfile ~/.ssh/id_ed25519
identityfile ~/.ssh/id_ed25519_sk
identityfile ~/.ssh/id_xmss
canonicaldomains none
globalknownhostsfile /etc/ssh/ssh_known_hosts /etc/ssh/ssh_known_hosts2
userknownhostsfile /Users/till/.ssh/known_hosts
sendenv LANG
sendenv LC_*
logverbose none
channeltimeout none
permitremoteopen any
addkeystoagent false
forwardagent no
connecttimeout none
tunneldevice any:any
canonicalizePermittedcnames none
controlpersist 600
escapechar ~
ipqos af21 cs1
rekeylimit 0 0
streamlocalbindmask 0177
syslogfacility USER'
++ awk '/^user / { print $2 }'
++ echo 'host 192.168.205.5
user root
hostname 192.168.205.5
port 22
addressfamily any
batchmode no
canonicalizefallbacklocal yes
canonicalizehostname false
checkhostip no
compression no
controlmaster true
enablesshkeysign no
clearallforwardings no
exitonforwardfailure no
fingerprinthash SHA256
forwardx11 no
forwardx11trusted no
gatewayports no
gssapiauthentication no
gssapidelegatecredentials no
hashknownhosts no
hostbasedauthentication no
identitiesonly no
kbdinteractiveauthentication yes
nohostauthenticationforlocalhost no
passwordauthentication yes
permitlocalcommand no
proxyusefdpass no
pubkeyauthentication true
requesttty auto
sessiontype default
stdinnull no
forkafterauthentication no
streamlocalbindunlink no
stricthostkeychecking ask
tcpkeepalive yes
tunnel false
verifyhostkeydns false
visualhostkey no
updatehostkeys true
enableescapecommandline no
canonicalizemaxdots 1
connectionattempts 1
forwardx11timeout 1200
numberofpasswordprompts 3
serveralivecountmax 2
serveraliveinterval 30
requiredrsasize 1024
obscurekeystroketiming yes
ciphers chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
controlpath /Users/till/.ssh/master-root@192.168.205.5:22
hostkeyalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
hostbasedacceptedalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
kexalgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
casignaturealgorithms ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
loglevel INFO
macs umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
securitykeyprovider internal
pubkeyacceptedalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
xauthlocation /usr/X11R6/bin/xauth
identityfile ~/.ssh/id_rsa
identityfile ~/.ssh/id_ecdsa
identityfile ~/.ssh/id_ecdsa_sk
identityfile ~/.ssh/id_ed25519
identityfile ~/.ssh/id_ed25519_sk
identityfile ~/.ssh/id_xmss
canonicaldomains none
globalknownhostsfile /etc/ssh/ssh_known_hosts /etc/ssh/ssh_known_hosts2
userknownhostsfile /Users/till/.ssh/known_hosts
sendenv LANG
sendenv LC_*
logverbose none
channeltimeout none
permitremoteopen any
addkeystoagent false
forwardagent no
connecttimeout none
tunneldevice any:any
canonicalizePermittedcnames none
controlpersist 600
escapechar ~
ipqos af21 cs1
rekeylimit 0 0
streamlocalbindmask 0177
syslogfacility USER'
+ ssh_user=root
++ echo 'host 192.168.205.5
user root
hostname 192.168.205.5
port 22
addressfamily any
batchmode no
canonicalizefallbacklocal yes
canonicalizehostname false
checkhostip no
compression no
controlmaster true
enablesshkeysign no
clearallforwardings no
exitonforwardfailure no
fingerprinthash SHA256
forwardx11 no
forwardx11trusted no
gatewayports no
gssapiauthentication no
gssapidelegatecredentials no
hashknownhosts no
hostbasedauthentication no
identitiesonly no
kbdinteractiveauthentication yes
nohostauthenticationforlocalhost no
passwordauthentication yes
permitlocalcommand no
proxyusefdpass no
pubkeyauthentication true
requesttty auto
sessiontype default
stdinnull no
forkafterauthentication no
streamlocalbindunlink no
stricthostkeychecking ask
tcpkeepalive yes
tunnel false
verifyhostkeydns false
visualhostkey no
updatehostkeys true
enableescapecommandline no
canonicalizemaxdots 1
connectionattempts 1
forwardx11timeout 1200
numberofpasswordprompts 3
serveralivecountmax 2
serveraliveinterval 30
requiredrsasize 1024
obscurekeystroketiming yes
ciphers chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
controlpath /Users/till/.ssh/master-root@192.168.205.5:22
hostkeyalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp3++ awk '/^hostname / { print $2 }'
84,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
hostbasedacceptedalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
kexalgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
casignaturealgorithms ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
loglevel INFO
macs umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
securitykeyprovider internal
pubkeyacceptedalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
xauthlocation /usr/X11R6/bin/xauth
identityfile ~/.ssh/id_rsa
identityfile ~/.ssh/id_ecdsa
identityfile ~/.ssh/id_ecdsa_sk
identityfile ~/.ssh/id_ed25519
identityfile ~/.ssh/id_ed25519_sk
identityfile ~/.ssh/id_xmss
canonicaldomains none
globalknownhostsfile /etc/ssh/ssh_known_hosts /etc/ssh/ssh_known_hosts2
userknownhostsfile /Users/till/.ssh/known_hosts
sendenv LANG
sendenv LC_*
logverbose none
channeltimeout none
permitremoteopen any
addkeystoagent false
forwardagent no
connecttimeout none
tunneldevice any:any
canonicalizePermittedcnames none
controlpersist 600
escapechar ~
ipqos af21 cs1
rekeylimit 0 0
streamlocalbindmask 0177
syslogfacility USER'
+ ssh_host=192.168.205.5
++ echo 'host 192.168.205.5
user root
hostname 192.168.205.5
port 22
addressfamily any
batchmode no
canonicalizefallbacklocal yes
canonicalizehostname false
checkhostip no
compression no
controlmaster true
enablesshkeysign no
clearallforwardings no
exitonforwardfailure no
fingerprinthash SHA256
forwardx11 no
forwardx11trusted no
gatewayports no
gssapiauthentication no
gssapidelegatecredentials no
hashknownhosts no
hostbasedauthentication no
identitiesonly no
kbdinteractiveauthentication yes
nohostauthenticationforlocalhost no
passwordauthentication yes
permitlocalcommand no
proxyusefdpass no
pubkeyauthentication true
requesttty auto
sessiontype default
stdinnull no
forkafterauthentication no
streamlocalbindunlink no
stricthostkeychecking ask
tcpkeepalive yes
tunnel false
verifyhostkeydns false
visualhostkey no
updatehostkeys true
enableescapecommandline no
canonicalizemaxdots 1
connectionattempts 1
forwardx11timeout 1200
numberofpasswordprompts 3
serveralivecountmax 2
serveraliveinterval 30
requiredrsasize 1024
obscurekeystroketiming yes
ciphers chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
controlpath /Users/till/.ssh/master-root@192.168.205.5:22
hostkeyalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
hostbasedacceptedalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
kexalgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
casignaturealgorithms ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
loglevel INFO
macs umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
securitykeyprovider internal
pubkeyacceptedalgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
xauthlocation /usr/X11R6/bin/xauth
identityfile ~/.ssh/id_rsa
identityfile ~/.ssh/id_ecdsa
identityfile ~/.ssh/id_ecdsa_sk
identityfile ~/.ssh/id_ed25519
identityfile ~/.ssh/id_ed25519_sk
identityfile ~/.ssh/id_xmss
canonicaldomains none
globalknownhostsfile /etc/ssh/ssh_known_hosts /etc/ssh/ssh_known_hosts2
userknownhostsfile /Users/till/.ssh/known_hosts
sendenv LANG
sendenv LC_*
logverbose none
channeltimeout none
permitremoteopen any
addkeystoagent false
forwardagent no
connecttimeout none
tunneldevice any:any
canonicalizePermittedcnames none
controlpersist 600
escapechar ~
ipqos af21 cs1
rekeylimit 0 0
streamlocalbindmask 0177
syslogfacility USER'
++ awk '/^port / { print $2 }'
+ ssh_port=22
+ step Uploading install SSH keys
+ echo '### Uploading install SSH keys ###'
### Uploading install SSH keys ###
+ [[ -n '' ]]
+ ssh-copy-id -i /tmp/tmp.VkXuGw8ixd/nixos-anywhere.pub -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@192.168.205.5
/etc/profiles/per-user/till/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/tmp/tmp.VkXuGw8ixd/nixos-anywhere.pub"
/etc/profiles/per-user/till/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/etc/profiles/per-user/till/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Warning: Permanently added '192.168.205.5' (ED25519) to the list of known hosts.
ControlSocket /Users/till/.ssh/master-root@192.168.205.5:22 already exists, disabling multiplexing

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh -o 'ConnectTimeout=10' -o 'UserKnownHostsFile=/dev/null' -o 'StrictHostKeyChecking=no' 'root@192.168.205.5'"
and check to make sure that only the key(s) you wanted were added.

+ step Gathering machine facts
+ echo '### Gathering machine facts ###'
### Gathering machine facts ###
+ import_facts
+ local facts filtered_facts
++ ssh_ -o ConnectTimeout=10 enable_debug=-x sh --
++ ssh -t -i /tmp/tmp.VkXuGw8ixd/nixos-anywhere -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@192.168.205.5 -o ConnectTimeout=10 enable_debug=-x sh --
Pseudo-terminal will not be allocated because stdin is not a terminal.
Warning: Permanently added '192.168.205.5' (ED25519) to the list of known hosts.
ControlSocket /Users/till/.ssh/master-root@192.168.205.5:22 already exists, disabling multiplexing
++ test -f /etc/os-release
++ grep -q ID=nixos /etc/os-release
++ echo y
+ is_nixos=y
+ cat
++ uname
++ uname -m
++ test -f /etc/is_kexec
++ echo n
++ '[' y = y ']'
++ grep -q VARIANT_ID=installer /etc/os-release
++ echo y
+++ has systemd-detect-virt
+++ command -v systemd-detect-virt
+++ echo y
++ '[' y = y ']'
++ systemd-detect-virt --container
+++ has ip
+++ command -v ip
+++ echo y
++ '[' y = n ']'
++ ip r g 1
++ echo n
++ has tar
++ command -v tar
++ echo y
++ has sudo
++ command -v sudo
++ echo y
++ has doas
++ command -v doas
++ echo n
++ has wget
++ command -v wget
++ echo n
++ has curl
++ command -v curl
++ echo y
++ has setsid
++ command -v setsid
++ echo y
+ facts='is_os=Linux
is_arch=x86_64
is_kexec=n
is_nixos=y
is_installer=y
is_container=none
has_ipv6_only=n
has_tar=y
has_sudo=y
has_doas=n
has_wget=n
has_curl=y
has_setsid=y'
++ echo 'is_os=Linux
is_arch=x86_64
is_kexec=n
is_nixos=y
is_installer=y
is_container=none
has_ipv6_only=n
has_tar=y
has_sudo=y
has_doas=n
has_wget=n
has_curl=y
has_setsid=y'
++ grep -E '^(has|is)_[a-z0-9_]+=\S+'
+ filtered_facts='is_os=Linux
is_arch=x86_64
is_kexec=n
is_nixos=y
is_installer=y
is_container=none
has_ipv6_only=n
has_tar=y
has_sudo=y
has_doas=n
has_wget=n
has_curl=y
has_setsid=y'
+ [[ -z is_os=Linux
is_arch=x86_64
is_kexec=n
is_nixos=y
is_installer=y
is_container=none
has_ipv6_only=n
has_tar=y
has_sudo=y
has_doas=n
has_wget=n
has_curl=y
has_setsid=y ]]
++ echo 'is_os=Linux
is_arch=x86_64
is_kexec=n
is_nixos=y
is_installer=y
is_container=none
has_ipv6_only=n
has_tar=y
has_sudo=y
has_doas=n
has_wget=n
has_curl=y
has_setsid=y'
++ xargs
+ export is_os=Linux is_arch=x86_64 is_kexec=n is_nixos=y is_installer=y is_container=none has_ipv6_only=n has_tar=y has_sudo=y has_doas=n has_wget=n has_curl=y has_setsid=y
+ is_os=Linux
+ is_arch=x86_64
+ is_kexec=n
+ is_nixos=y
+ is_installer=y
+ is_container=none
+ has_ipv6_only=n
+ has_tar=y
+ has_sudo=y
+ has_doas=n
+ has_wget=n
+ has_curl=y
+ has_setsid=y
+ [[ y == \n ]]
+ [[ y == \n ]]
+ maybe_sudo=
+ [[ y == \y ]]
+ maybe_sudo=sudo
+ [[ Linux != \L\i\n\u\x ]]
+ [[ n == \n ]]
+ [[ y == \n ]]
+ [[ y == \y ]]
+ [[ root != \r\o\o\t ]]
+ [[ n == \y ]]
+ [[ -n /nix/store/hhdz1qxvr1dbyfy99i8c0qx6a1brvq91-disko ]]
+ nix_copy --to ssh://root@192.168.205.5 /nix/store/hhdz1qxvr1dbyfy99i8c0qx6a1brvq91-disko
+ NIX_SSHOPTS='-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /tmp/tmp.VkXuGw8ixd/nixos-anywhere '
+ nix copy --extra-experimental-features 'nix-command flakes' --no-write-lock-file -L --substitute-on-destination --to ssh://root@192.168.205.5 /nix/store/hhdz1qxvr1dbyfy99i8c0qx6a1brvq91-disko
Warning: Permanently added '192.168.205.5' (ED25519) to the list of known hosts.
ControlSocket /Users/till/.ssh/master-root@192.168.205.5:22 already exists, disabling multiplexing

It seems that the disko script is copied to the target machine, but then never executed or silently fails and stops everything. dmesg and journalctl on the target machine also don't seem to have any error messages.

I am quite new to nix and nixos so I am probably missing something quite obvious, but after going through all the documentation and examples I found, I can't figure out what.

Thanks in advance :)

sedlund commented 2 months ago

It seems that the disko script is copied to the target machine

you verified that its there in the store on the remote?

I would suggest on the target machine try using disko-install directly and see if that works - then work backwards.

dereulenspiegel commented 2 months ago

Hi @sedlund and thanks for the advice. Thanks to your hint I think the problem is the nix copy command stalling after calling nix-store --serve --write on the target. Running the nix copy command manually and with debug gives me

performing daemon worker op: 40
querying info about missing paths...
starting pool of 9 threads
performing daemon worker op: 46
substitution of '/nix/store/y4pn8qg3vjf4gmvg3s1i6z79kpb2cwmw-disko': created
substitution of '/nix/store/y4pn8qg3vjf4gmvg3s1i6z79kpb2cwmw-disko': woken up
querying info about missing paths...
starting pool of 9 threads
entered goal loop
substitution of '/nix/store/y4pn8qg3vjf4gmvg3s1i6z79kpb2cwmw-disko': init
acquiring write lock on '/nix/var/nix/temproots/56502'
substitution of '/nix/store/y4pn8qg3vjf4gmvg3s1i6z79kpb2cwmw-disko': done
substitution of '/nix/store/y4pn8qg3vjf4gmvg3s1i6z79kpb2cwmw-disko': goal destroyed
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
performing daemon worker op: 26
OpenSSH_9.7p1, OpenSSL 3.0.14 4 Jun 2024
debug1: Reading configuration data /Users/till/.ssh/config
debug1: /Users/till/.ssh/config line 1: include ~/.ssh/config.d/* matched no files
debug1: /Users/till/.ssh/config line 15: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/100-linux-builder.conf
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Connecting to 192.168.205.5 [192.168.205.5] port 22.
debug1: Connection established.
debug1: identity file /Users/till/.ssh/id_rsa type -1
debug1: identity file /Users/till/.ssh/id_rsa-cert type -1
debug1: identity file /Users/till/.ssh/id_ecdsa type -1
debug1: identity file /Users/till/.ssh/id_ecdsa-cert type -1
debug1: identity file /Users/till/.ssh/id_ecdsa_sk type -1
debug1: identity file /Users/till/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /Users/till/.ssh/id_ed25519 type -1
debug1: identity file /Users/till/.ssh/id_ed25519-cert type -1
debug1: identity file /Users/till/.ssh/id_ed25519_sk type -1
debug1: identity file /Users/till/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /Users/till/.ssh/id_xmss type -1
debug1: identity file /Users/till/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_9.7
debug1: compat_banner: match: OpenSSH_9.7 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.205.5:22 as 'root'
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: sntrup761x25519-sha512@openssh.com
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:z5UzX+CiDaXVWEUmP3cKkWfMlCCMVY0Pq5R03kSmzyw
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '192.168.205.5' is known and matches the ED25519 host key.
debug1: Found key in /Users/till/.ssh/known_hosts:23
debug1: ssh_packet_send2_wrapped: resetting send seqnr 3
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: Sending SSH2_MSG_EXT_INFO
debug1: expecting SSH2_MSG_NEWKEYS
debug1: ssh_packet_read_poll2: resetting read seqnr 3
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_ext_info_client_parse: server-sig-algs=<ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256>
debug1: kex_ext_info_check_ver: publickey-hostbound@openssh.com=<0>
debug1: kex_ext_info_check_ver: ping@openssh.com=<0>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_ext_info_client_parse: server-sig-algs=<ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256>
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: get_agent_identities: agent returned 1 keys
debug1: Will attempt key: cardno:8_710_580 RSA SHA256:C5Sfgoq8w2laHK7tPE5CdXezdAlhOtMemNuVuPdW1Ig agent
debug1: Will attempt key: /Users/till/.ssh/id_rsa
debug1: Will attempt key: /Users/till/.ssh/id_ecdsa
debug1: Will attempt key: /Users/till/.ssh/id_ecdsa_sk
debug1: Will attempt key: /Users/till/.ssh/id_ed25519
debug1: Will attempt key: /Users/till/.ssh/id_ed25519_sk
debug1: Will attempt key: /Users/till/.ssh/id_xmss
debug1: Offering public key: cardno:8_710_580 RSA SHA256:C5Sfgoq8w2laHK7tPE5CdXezdAlhOtMemNuVuPdW1Ig agent
debug1: Server accepts key: cardno:8_710_580 RSA SHA256:C5Sfgoq8w2laHK7tPE5CdXezdAlhOtMemNuVuPdW1Ig agent
Authenticated to 192.168.205.5 ([192.168.205.5]:22) using "publickey".
debug1: setting up multiplex master socket
ControlSocket /Users/till/.ssh/master-root@192.168.205.5:22 already exists, disabling multiplexing
debug1: channel 0: new session [client-session] (inactive timeout: 0)
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: exec
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: client_input_hostkeys: searching /Users/till/.ssh/known_hosts for 192.168.205.5 / (none)
debug1: client_input_hostkeys: no new or deprecated keys from server
debug1: Remote: /etc/ssh/authorized_keys.d/root:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: Remote: /etc/ssh/authorized_keys.d/root:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: Sending environment.
debug1: channel 0: setting env LC_TERMINAL_VERSION = "3.4.16"
debug1: channel 0: setting env LANG = "de_DE.UTF-8"
debug1: channel 0: setting env LC_TERMINAL = "iTerm2"
debug1: Sending command: nix-store --serve --write
debug1: pledge: fork

Which still doesn't contain any obvious error. I can also confirm that nix-store --serve --write is actually running on the target machine. Currently I am guessing that since nix-store --serve works by serving through stdin/stdout that maybe my ssh config could interfering with this process, causing a stall. But maybe I am wrong, so I am still glad for every pointer in more directions helping me debug/fix this :)

sedlund commented 2 months ago

It looks like you have a yubikey or using macos touchid.

the last output with your manual run of nix copy shows nix-copy is authenticating with it.

I would suggest setting a root password on the target and not have any keys on the remote and typing it in when running nixos-anywhere. It will then use its self generated key for connecting.

dereulenspiegel commented 2 months ago

Ok, after experimenting with my SSH config I found the cause for the problem (but not an explanation). By default I set ControlMaster yes for all my hosts. This seems to cause problems. Temporarely setting ControlMaster no for the target host seem to solve the problem. Thanks for the advice and I hope this issue helps anyone else struggling with this :)