Nixops locks itself out of virtualbox guests on reboot

[berdario]

I guess this could be related to this issue, still it'd be nice if there was an easy way to work around this.

Failing that, this should at least be documented in the manual, so that the suggested example configuration for deployments on VirtualBox will statically specify ip addresses.

this is the dev.nix of (which is pretty much the example trivial-vbox.nix from the manual):

{
    network.description = "dev";

    devmachine = { config, pkgs, ... }:
    {
        environment.systemPackages = with pkgs; [
          socat
        ];
        deployment.targetEnv = "virtualbox";
        deployment.virtualbox.memorySize = 2000; # megabytes
    } ;
}

The machine was originally deployed just fine, on 192.168.56.102, this is the partial output of nixops export -d dev:

{
  "72421978-e769-11e4-a07e-34e6d704a2e9": {
    "configsPath": "/nix/store/khfc6iqvgps30lx3nlw6j6rpna41xma1-nixops-machines", 
    "description": "dev", 
    "name": "dev", 
    "nixExprs": "[\"/home/dario/Projects/nixmachines/dev.nix\"]", 
    "resources": {
      "devmachine": {
        "configsPath": "/nix/store/khfc6iqvgps30lx3nlw6j6rpna41xma1-nixops-machines", 
        "index": "0", 
        "keys": "{}", 
        "privateIpv4": "192.168.56.102", 
        "publicVpnKey": "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEGIBeQMwuDyHYL7V+2fJSWgAXr8NaPgvbdP4kfLojVzlTYWnUxcg/VNvbbNhsH68/uCLhly5hPkbQoUM/cy2Q= NixOps VPN key of devmachine", 
        "sshPinged": "1", 
        "state": "3", 
        "storeKeysOnMachine": "0", 
        "toplevel": "/nix/store/kvj6vak0jpykp22dr9icqia8ca7v62ny-nixos-15.05pre59184.e07ea5c", 
        "type": "virtualbox", 
        "virtualbox.disks": "{\"disk1\": {\"path\": \"/home/dario/.VirtualBoxVMs/nixops-72421978-e769-11e4-a07e-34e6d704a2e9-devmachine/disk1.vdi\", \"attached\": true, \"port\": 0, \"created\": true}}", 
        "virtualbox.sataControllerCreated": "1", 
        "vmId": "nixops-72421978-e769-11e4-a07e-34e6d704a2e9-devmachine"
      }
    }
  }
}

and this is the output of VBoxManage guestproperty get nixops-72421978-e769-11e4-a07e-34e6d704a2e9-devmachine /VirtualBox/GuestInfo/Net/1/V4/IP:

Value: 192.168.56.101

If I manually connect with ssh, add 192.168.56.102 as another address, redeploy and then reboot I keep getting the new 192.168.56.101 address.

Even if I add deployment.targetHost = "192.168.56.101"; to the machine configuration, it'll get ignored since nixops still thinks that the machine is accessible on the old address...

I now worked around this by manually ssh into it, adding the old address, add the following to the configuration:

networking.interfaces.eth1.ip4 = [ { address = "192.168.56.101"; prefixLength = 24; } ];

redeploying, and now even after reboot it sticks...

Weirdly, if I try to change it back with

networking.interfaces.eth1.ip4 = [ { address = "192.168.56.102"; prefixLength = 24; } ];

The change is not picked up correctly by nixops, even after the successfull deploy, and will thus locks itself out again

[edolstra]

Yeah, it's very annoying that the IP addresses assigned by VirtualBox's DHCP server changes so quickly. The only workaround at the moment is to run "nixops deploy --check" - that should detect the new IP address.

[3noch]

@edolstra I've found that nixops deploy --check is not always enough. I often need to remove the offending IP from ~/.ssh/known_hosts as well.

[kirelagin]

Hm, what is the issue exactly? I believe the IP is synchronized on every start, so is the issue that it is changing while the machine is running?

Won’t updating the IP before each ssh (or even make private_ipv4 a property that’s updated on every read) solve this?

[jpotier]

It can be workable with ~/.ssh/config set to:

Host 192.168.0.*
  StrictHostKeyChecking no
  UserKnownHostsFile=/dev/null

AND nixops deploy --check

Still, that's just a workaround.

[robinp]

Note: --check didn't help me, but manually updating the IP address in the nixops state helped:

$ sqlite3 ~/.nixops/deployments.nixops then sqlite3> UPDATE ResourceAttrs SET value = '192.168.56.102' WHERE machine=6 AND name='privateIpv4';

[onixie]

Is it possible to allow user specify the network.privateIPv4 options and reconfigure guest os use static ip address? I think vagrant has some behavior like this. It is ok to use dhcp as the initial method to access the guest os, after that a predictable static ip address is more reasonable.