ppggff / vagrant-qemu

Use Vagrant to manage machines using QEMU. Test with Apple Silicon / M1 and CentOS aarch64 image
MIT License
408 stars 32 forks source link

can't force local IP #36

Open UriZafrir opened 1 year ago

UriZafrir commented 1 year ago

Hi all I am having the same issue as #33 but the solution offered isn't working for me. this is my vagrantfile: but the vms get the ip 127.0.0.1 when booting, 10.0.2.15 when typing ifconfig. I had to change the ssh port. Can anyone please assist?

Vagrant.configure("2") do |config|
  config.vm.provision "shell", inline: "echo Hello"

  config.vm.define "controlplane" do |cp|
    cp.vm.box = "perk/ubuntu-2204-arm64"
    cp.vm.hostname = "controlplane"
    #cp.vm.network :private_network, type: "dhcp", ip: "192.168.51.10"
    cp.vm.provider "qemu" do |qe|
      qe.memory = "2G"
      qe.ssh_port = "50022"
      qe.extra_netdev_args = "net=192.168.51.0/24,dhcpstart=192.168.51.10"
    end
  end

  config.vm.define "worker1" do |w1|
    w1.vm.box = "perk/ubuntu-2204-arm64"
    w1.vm.hostname = "worker1"
    #w1.vm.network :private_network, type: "dhcp", ip: "192.168.51.11"
    w1.vm.provider "qemu" do |qe|
      qe.memory = "2G"
      qe.ssh_port = "50023"
      qe.extra_netdev_args = "net=192.168.51.0/24,dhcpstart=192.168.51.11"
    end
  end
end

image image

ppggff commented 1 year ago

I tried with your config, it works with the correct ip.

❯ vagrant ssh worker1
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-57-generic aarch64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Mon Apr 17 07:52:33 UTC 2023

  System load:           0.1181640625
  Usage of /:            2.5% of 61.84GB
  Memory usage:          10%
  Swap usage:            0%
  Processes:             105
  Users logged in:       0
  IPv4 address for eth0: 192.168.51.11
  IPv6 address for eth0: fec0::5054:ff:fe12:3456

❯ vagrant ssh controlplane
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-57-generic aarch64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Mon Apr 17 07:54:30 UTC 2023

  System load:           0.013671875
  Usage of /:            2.5% of 61.84GB
  Memory usage:          10%
  Swap usage:            0%
  Processes:             106
  Users logged in:       0
  IPv4 address for eth0: 192.168.51.10
  IPv6 address for eth0: fec0::5054:ff:fe12:3456

You can you ps -ef|grep qemu to confirm that the network setting is in the command line of qemu, for example:

... /opt/homebrew/bin/qemu-system-aarch64 ... -netdev user,id=net0,hostfwd=tcp::50024-:22,net=192.168.51.0/24,dhcpstart=192.168.51.10 -drive if=virtio,format=qcow2 ...
benjaminfayaz commented 1 year ago

I seem to encounter a similar issue 👀

I am using this example block from the README:

Vagrant.configure("2") do |config|
  config.vm.box = "perk/ubuntu-20.04-arm64" # README uses debian/bullseye64

  config.vm.provider "qemu" do |qe|
    qe.extra_netdev_args = "net=192.168.51.0/24,dhcpstart=192.168.51.10"
  end
end
Startup log ``` Bringing machine 'default' up with 'qemu' provider... ==> default: Checking if box 'perk/ubuntu-20.04-arm64' version '20230117' is up to date... ==> default: Warning! The QEMU provider doesn't support any of the Vagrant ==> default: high-level network configurations (`config.vm.network`). They ==> default: will be silently ignored. ==> default: Starting the instance... ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:50022 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Connection reset. Retrying... default: Warning: Remote connection disconnect. Retrying... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... ==> default: Machine booted and ready! ```

However if I try to ping or SSH into it, it does not work:

ping and ssh ``` > ping 192.168.51.10 PING 192.168.51.10 (192.168.51.10): 56 data bytes Request timeout for icmp_seq 0 Request timeout for icmp_seq 1 [...] > ssh -o ConnectTimeout=5 -o ConnectionAttempts=1 vagrant@192.168.51.10 ssh: connect to host 192.168.51.10 port 22: Operation timed out ```

vagrant ssh works fine and ifconfig shows the correct ip address

vagrant ssh and ifconfig ``` > vagrant ssh Welcome to Ubuntu 20.04.5 LTS (GNU/Linux 5.4.0-137-generic aarch64) [...] # inside the VM > ifconfig eth0: flags=4163 mtu 1500 inet 192.168.51.10 netmask 255.255.255.0 broadcast 192.168.51.255 [...] ```

Also the running qemu proccess looks fine to me:

qemu-system proccess ``` # On host system > ps -ef | grep qemu-system /opt/homebrew/bin/qemu-system-aarch64 -machine virt,accel=hvf,highmem=on -cpu host -smp 2 -m 4G -device virtio-net-device,netdev=net0 -netdev user,id=net0,hostfwd=tcp::50022-:22,net=192.168.51.0/24,dhcpstart=192.168.51.10 -drive if=virtio,format=qcow2 [...] ```

System information

Machine: Mac M1 Pro OS: Ventura 13.3 Vagrant version: Vagrant 2.3.4

Happy to provide additional infomation 😃

ppggff commented 1 year ago

This is the expected behavior.

The ip address in the configuration file (192.168.51.10) is the internal address of the virtual machine. It's works like behind a NAT network. The port 22 is mapped as 50022 on the host (qe.ssh_port = "50022"), so vagrant ssh works.

To get more information about network of qemu, https://wiki.qemu.org/Documentation/Networking, we use user network by default.

Currently, this plugin doesn't support complex network config with vagrant gramma, but you can append qemu config to achieve same effect. What's your requirement? Maybe I could add more example or figure out new feature for that.

benjaminfayaz commented 1 year ago

Oh, thank you for clarifying. What I am trying to achieve is making the VM run on a specific IP within my private network so I can access it by connecting to that IP from my host machine. Is that somehow possible?

EDIT: When using the virtualbox provider, I was able to achieve this by setting

config.vm.network "private_network", ip: "192.168.51.10"

However with this provider I discarded this due to the following warning during the startup log:

==> default: Warning! The QEMU provider doesn't support any of the Vagrant
==> default: high-level network configurations (`config.vm.network`). They
==> default: will be silently ignored.
UriZafrir commented 1 year ago

Hi I am also on Mac M1. I am trying to bootstrap a Kubernetes cluster with one control plane and one worker node. @benjaminfayaz is getting the correct ip in the ifconfig command but I am not. could someone kindly explain what I am missing?

image

this is the output of ps aux | grep qemu. it seems the net=192.168.51.0/24,dhcpstart=192.168.51.10 part is not there for some reason.

/opt/homebrew/bin/qemu-system-aarch64 -machine virt,accel=hvf,highmem=on -cpu host -smp 2 -m 2G -device virtio-net-device,netdev=net0 -netdev user,id=net0,hostfwd=tcp::50022-:22 -drive if=virtio,format=qcow2,file=/Users//Tmp/vagrant/.vagrant/machines/controlplane/qemu/88gicSkM6KE/linked-box.img -drive if=pflash,format=raw,file=/Users//Tmp/vagrant/.vagrant/machines/controlplane/qemu/88gicSkM6KE/edk2-aarch64-code.fd,readonly=on -drive if=pflash,format=raw,file=/Users//Tmp/vagrant/.vagrant/machines/controlplane/qemu/88gicSkM6KE/edk2-arm-vars.fd -chardev socket,id=mon0,path=/Users//.vagrant.d/tmp/vagrant-qemu/88gicSkM6KE/qemu_socket,server=on,wait=off -mon chardev=mon0,mode=readline -chardev socket,id=ser0,path=/Users//.vagrant.d/tmp/vagrant-qemu/88gicSkM6KE/qemu_socket_serial,server=on,wait=off -serial chardev:ser0 -pidfile /Users//Tmp/vagrant/.vagrant/machines/controlplane/qemu/88gicSkM6KE/qemu.pid -parallel null -monitor none -display none -vga none -daemonize /usr/bin/ssh vagrant@127.0.0.1 -p 50022 -o LogLevel=FATAL -o Compression=yes -o DSAAuthentication=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /Users//Tmp/vagrant/.vagrant/machines/controlplane/qemu/private_key /opt/homebrew/bin/qemu-system-aarch64 -machine virt,accel=hvf,highmem=on -cpu host -smp 2 -m 2G -device virtio-net-device,netdev=net0 -netdev user,id=net0,hostfwd=tcp::50023-:22 -drive if=virtio,format=qcow2,file=/Users//Tmp/vagrant/.vagrant/machines/worker1/qemu/NMO8p1jyed4/linked-box.img -drive if=pflash,format=raw,file=/Users//Tmp/vagrant/.vagrant/machines/worker1/qemu/NMO8p1jyed4/edk2-aarch64-code.fd,readonly=on -drive if=pflash,format=raw,file=/Users//Tmp/vagrant/.vagrant/machines/worker1/qemu/NMO8p1jyed4/edk2-arm-vars.fd -chardev socket,id=mon0,path=/Users//.vagrant.d/tmp/vagrant-qemu/NMO8p1jyed4/qemu_socket,server=on,wait=off -mon chardev=mon0,mode=readline -chardev socket,id=ser0,path=/Users//.vagrant.d/tmp/vagrant-qemu/NMO8p1jyed4/qemu_socket_serial,server=on,wait=off -serial chardev:ser0 -pidfile /Users//Tmp/vagrant/.vagrant/machines/worker1/qemu/NMO8p1jyed4/qemu.pid -parallel null -monitor none -display none -vga none -daemonize grep qemu

UriZafrir commented 1 year ago

As I see from my previous comment for some reason qemu won't accept the arguments for the private IP. Any ideas please?

ppggff commented 1 year ago

@UriZafrir What version of the plugin do you use? And please try a full restart?

UriZafrir commented 1 year ago

@ppggff found I was using 0.19. updated to 0.3.4. now working! thanks so much :))

image
UriZafrir commented 1 year ago

another question please. As I understand as long as I don't use tap networking or socket I can't make the two communicate (Except port forwarding which I think is not the solution I am looking for) for example this: https://github.com/ppggff/vagrant-qemu/issues/33

So now I am trying to create a TAP network:

Vagrant.configure("2") do |config|
    (2..3).each do |i|
        config.vm.define "vm-#{i}" do |web|
        web.vm.box = "perk/ubuntu-2204-arm64"
        if i==2
            web.vm.hostname = "master-node"  
        end
        if i==3
            web.vm.hostname = "worker1"
        end
        web.vm.network "private_network", type: "dhcp", ip: "192.168.51.#{i}"
        web.vm.provider "qemu" do |qe|
            qe.memory = "2G"
            qe.net_device = "e1000"
            qe.drive_interface = "ide"
            qe.extra_netdev_args = "net=192.168.51.0/24,dhcpstart=192.168.51.#{i}"
        end
    end
end
end

I would appreciate your assistance.

this is the error I am getting:

Command: ["qemu-system-aarch64", "-machine", "virt,accel=hvf,highmem=on", "-cpu", "host", "-smp", "2", "-m", "2G", "-device", "e1000,netdev=net0", "-netdev", "user,id=net0,hostfwd=tcp::50022-:22,net=192.168.51.0/24,dhcpstart=192.168.51.3", "-drive", "if=ide,format=qcow2,file=/Users/urizafrir/Tmp/vagrant/.vagrant/machines/vm-3/qemu/_0-vRG5wfuI/linked-box.img", "-drive", "if=pflash,format=raw,file=/Users/urizafrir/Tmp/vagrant/.vagrant/machines/vm-3/qemu/_0-vRG5wfuI/edk2-aarch64-code.fd,readonly=on", "-drive", "if=pflash,format=raw,file=/Users/urizafrir/Tmp/vagrant/.vagrant/machines/vm-3/qemu/_0-vRG5wfuI/edk2-arm-vars.fd", "-chardev", "socket,id=mon0,path=/Users/urizafrir/.vagrant.d/tmp/vagrant-qemu/_0-vRG5wfuI/qemu_socket,server=on,wait=off", "-mon", "chardev=mon0,mode=readline", "-chardev", "socket,id=ser0,path=/Users/urizafrir/.vagrant.d/tmp/vagrant-qemu/_0-vRG5wfuI/qemu_socket_serial,server=on,wait=off", "-serial", "chardev:ser0", "-pidfile", "/Users/urizafrir/Tmp/vagrant/.vagrant/machines/vm-3/qemu/_0-vRG5wfuI/qemu.pid", "-parallel", "null", "-monitor", "none", "-display", "none", "-vga", "none", "-daemonize", {:notify=>[:stdout, :stderr, :stdin]}]

Stderr: qemu-system-aarch64: -netdev user,id=net0,hostfwd=tcp::50022-:22,net=192.168.51.0/24,dhcpstart=192.168.51.3: DHCP must be different from host and DNS

ppggff commented 1 year ago

see #40 for future develop of private network

smitjainsj commented 9 months ago

@ppggff Is there is way to have two network card's configured on the vm.

Whenever I try to add a network via vagrant file, it overrides the eth0 and configures the 192.168.x.x range.

nodes = [
    { 
        :hostname => 'root',
        :net => '192.168.51.0/24',
        :dhcp => '192.168.51.10',
        :mem => '2G',
        :cpus => '2'
    },
]

Vagrant.configure(2) do |config|
    nodes.each do |node|
        config.vm.define node[:hostname] do |nodeconfig|
            nodeconfig.vm.box_check_update = false
            # nodeconfig.vm.synced_folder '.', '/vagrant', disabled: true
            nodeconfig.vm.synced_folder ".", "/vagrant", type: "smb", smb_host: '192.168.34.10', smb_username: "xxx@xxx", smb_password: "xxxx"
            nodeconfig.vm.box = 'ppggff/centos-7-aarch64-2009-4K'
            nodeconfig.vm.hostname = node[:hostname] + ".example.com"
            # nodeconfig.vm.provision :shell, :path => "bootstrap.sh"
            nodeconfig.vm.provider :qemu do |qe|
                qe.memory = node[:mem]
                qe.smp = "cores=#{node[:cpus]}"
                qe.qe.net_device = "virtio-net-device,netdev=net1"
                qe.extra_netdev_args = "id=net1,net=#{node[:net]},dhcpstart=#{node[:dhcp]}"
            end
        end
    end
end

Also, if you notice the SMB configuration also doesn't work and results in the permission denied error [1].

1 ``` ❯ vagrant up Bringing machine 'root' up with 'qemu' provider... ==> root: Preparing SMB shared folders... ==> *****: Warning! The QEMU provider doesn't support any of the Vagrant ==> *****: high-level network configurations (`config.vm.network`). They ==> *****: will be silently ignored. ==> *****: Starting the instance... ==> *****: Waiting for machine to boot. This may take a few minutes... *****: SSH address: 127.0.0.1:50022 *****: SSH username: vagrant *****: SSH auth method: private key *****: *****: Vagrant insecure key detected. Vagrant will automatically replace *****: this with a newly generated keypair for better security. *****: *****: Inserting generated public key within guest... *****: Removing insecure key from the guest if it's present... *****: Key inserted! Disconnecting and reconnecting using new SSH key... ==> *****: Machine booted and ready! ==> *****: Setting hostname... ==> *****: Mounting SMB shared folders... *****: /xxxxx/xxxx => /vagrant Failed to mount folders in Linux guest. This is usually because the "vboxsf" file system is not available. Please verify that the guest additions are properly installed in the guest and can work properly. The command attempted was: mount -t cifs -o sec=ntlmssp,nounix,noperm,credentials=/etc/smb_creds_vgt-540667cace0c3b255bae4b0cc6c94b88-6ad5fdbcbf2eaa93bd62f92333a2e6e5,uid=1001,gid=1001,mfsymlinks,_netdev,nofail //192.168.34.10/vgt-540667cace0c3b255bae4b0cc6c94b88-6ad5fdbcbf2eaa93bd62f92333a2e6e5 /vagrant The error output from the last command was: mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) ```

I think the thread is here for this conversation - https://github.com/ppggff/vagrant-qemu/issues/40

ppggff commented 9 months ago

@smitjainsj yes, you can:

Some notes about network in qemu:

For the vmnet config #40, I will try to give a workable vagrant config with extra_qemu_args soon.

About the SMB error, please try the vagrant doc https://developer.hashicorp.com/vagrant/docs/synced-folders/smb