search

GNS3 / gns3-gui

GNS3 Graphical Network Simulator
http://www.gns3.com
GNU General Public License v3.0
2.17k stars 436 forks source link

GNS3 VM for KVM hypervisor based #2818

Closed mabed-fr closed 5 years ago

mabed-fr commented 5 years ago

Hello, I use KVM hypervisor but i don't have a image for this.

Can you add ?

thanl's

grossmj commented 5 years ago

I understand you would like to have a GNS3 VM image for KVM. Just like we distribute GNS3 VM images for VMware and VirtualBox. Is this correct?

mabed-fr commented 5 years ago

Yes is correct !

It's possible for the new version ?

qcow2 is kvm

grossmj commented 5 years ago

This shouldn't be too hard to do. Let me check if we can do this for version 2.2

mabed-fr commented 5 years ago

Parfait !

grossmj commented 5 years ago

The easiest thing for now is to download the GNS3 VM for VirtualBox and convert the disks (.vdi files) to qcow2 using qemu-img

qemu-img convert -f vdi -O qcow2 disk-1.vdi disk-2.qcow2

jaroslaw-bazydlo commented 5 years ago

@grossmj Do you see any performance advantages when using VBox VDI disks compared to VMWare VMDKs for QEMU/KVM (Now I notices I cant use VIRTIO in template definition)? Are GNS3 VM somehow optimized for specific hypervisor platform. I am preparing myself to JNCIE-SEC and now researching to put whole lab infrastructure into GNS3. I have already managed to convert VMWare version of VM and import them into Proxmox (Enterprise-class Virtual Infrastructure Managed based on QEMU/KVM). Unfortunately I see performance degradation during Juniper vSRX3.0 boot (19.1R1) inside converted GNS3-VM when compared to vSRX running on the top pf Proxmox. I was wandering what could cause this? Nested Virtualization? Disk speed degradation when running QCOW2 inside QCOW2 disk image of vSRX3.0 runnung under control of GNS3? I am trying to use 2.2rc3 version in my test. I am also very interested in having KVM version of GNS3 VM. People are using Linux as their workstation more often these days. That would be really great to run GNS3 VM with Boxes/VIRT-MANAGER/VIRSH so I am very glad you put this in milestone for next version of GNS3. I wish I have this in current but it better to have something than nothing in the future.

jaroslaw-bazydlo commented 5 years ago

It looks like GNS3 VM for Virtualbox is distributed with VMDK disks (which are supported in VBox). So the question is if GNS3VM for different HVs are different or exactly the same?

I did some more research today and it seems that nesting KVM guests costs performance. That is understood but according to RedHat's https://www.redhat.com/en/blog/inception-how-usable-are-nested-kvm-guests nested guests level 1 and 2 should be comparable.

My numbers are quite different with Juniper vSRX 19.2R1 .qcow2 image:

  1. PVE 5.3 - VSRX3 19.2R1(local storage) - 2m59s
  2. PVE 5.3 - Ubuntu 18.04LTS/KVM(local storage) - VSRX3 19.2R1 - 13M25S
  3. PVE 5.3 - Ubuntu 18.04LTS/KVM(nfs storage) - VSRX3 19.2R1 - 13m19s
  4. PVE 5.3 - GNS3 VM for VMWare/Converted (nfs storage) - VSRX3 19.2R1 - 26m15s
  5. ESXi 6.7 - GNS3 VM for VMWare 2.2rc3 (local storage) - VSRX3 19.2R1 - 4m16s

I did all tests having KVM acceleration enabled, KVM nesting enabled, ACPIv disabled, PML disabled and finally QEMU KSM disabled. A small improvements when added "pti=off kpti=off spectre_v2=off" in /etc/default/grub and updated GRUB2 but it was really 2-3 minutes better that 14 minutes in plain PVE - Ubuntu 18.04/KVM - Juniper vSRX cases.

I spent the whole day looking at GNS3VM settings but could not really find reasons why it was the worst case here. IT SHOULD BE SIMILAR TO MY UBUNTU/KVM CASES. I thought NFS could be the problem but checked that it is not. I think I will stay with gns3-server installed for the time being on the top of Ubuntu18.04/KVM running on PVE platform until I find why GNS3VM boots my vSRX so slowly.

I was really surprised GNS3VM on ESXi behaved so well (I cant use it since borrow space for tests only). That is why I started to think what differs GNS3VM for different HV platforms. I need to investigate it further to have my eight vSRX3.0 devices of JNCIE-SEC lab infrastructure easy to use with fabolous GNS3 ennvironment :-)

Regards J.

jaroslaw-bazydlo commented 5 years ago

Finally managed to achive 14 minutes in scenario number 3:

  1. PVE 5.3 - GNS3 VM for VMWare/Converted (nfs storage) - VSRX3 19.2R1

Conditions:

  1. Used GNS3 VM for VirtualBox and its VMDKs disks (converter to QCOW2 before imported them to PVE).
  2. Running GNS disks as Virtio-BLK (needed to modify /etc/fstab to mount /opt by UUID rather than /dev/sdb1 device name since they are recognized as /dev/vda1 and /dev/vdb1).
  3. Changed Disk interface as Virtio for HDA (Primaty Master).

So happy to achive results comparable to Ubuntu/KVM since I can stay with GNS3VM but still cannot believe why nesting VMs costs so much in term of performance (at least x2/x2,5 slower on guest level 1).

Regards J.

mabed-fr commented 5 years ago

Do you add native disk or config for kvm for the latest version ?

jaroslaw-bazydlo commented 5 years ago

Yes I did. My KVM GNE3VM has Virtio Blk disk and Virtio network adapters which guaranties best possible performance. Since GNS3 is build on the top of 18.04 LTS you have all drivers build in the kernel. Now there is a small drawback. You have to have SCSI/ATA hardisks configured when you first starting GNS3VM since there are scripts winthin the initialization process which uses SD as the name of the disk. Virtio Blk devices is recognized as VDA in Linux kernel.

So you should first define GNS3VM as IDE or SCSI and run it (to let all initalization scripts do its work then log as root in shell and modify /etc/fstab NOT to use /dev/sdb1 to mount /opt.

My procedure:

  1. Use "blockid" to find UUID for /dev/sdb1 partition
  2. Use this UUID to m ount /opt (you ca use root partition from /etc/fstab as an example).
  3. Shut down GNS3VM and change controller type in KVM guest definition from SCSI/ATA to Virtio.
  4. Start GNS3VM. This works as a charm and you have better performance too.

I think GNS3 guys will have to modify a couple of scripts (I was not looking for all places where SDB is sued) when they decide to support GNS3VM officially. They could even add it to 2.2 :-) since doing that is really trivial and would take a hour or two to find all SDB dependencied in scripts they have.

Let's keep the fingers crossed!

Regards J.

grossmj commented 5 years ago

@jaroslaw-bazydlo thanks for all your findings! We will use the UUID for /dev/sdb1 in /etc/fstab

Let us know if you think there are other improvements we can do in the GNS3 VM.

grossmj commented 5 years ago

The GNS3 VM for KVM hypervisor will be released with our next GNS3 version 2.2 release.

mabed-fr commented 5 years ago

The GNS3 VM for KVM hypervisor will be released with our next GNS3 version 2.2 release.

Can you add with the disk the "best config" for kvm ?

jaroslaw-bazydlo commented 5 years ago

Mathieu, I am using Linux/KVM in different favours A LOT these days.

For single hosted type of use I prefer QEMU/KVM managed by libvirt via Virt-manager/Virsh (I guess this is what people do in most cases). For QEMU/KVM enterprise clusters we are using Proxmox PVE which manages its VMs by itself without intermediate layer (just like GNS3-server does BTW).

But I think there some general rules the one must keep in order to gain maximum performance.

  1. vCPU type: host (on workstation use balooning as it saves RAM).
  2. vHDD disks Bus type: Virtio-BPK (I was experimenting with cache mode but it does not seem to influence performance much, I plan to investigate that later).
  3. vNIC type: Virtio
  4. vRAM 8-16G+ (depends on how much will be consumed by your VMs, balooning helps here a lot and is supported in Linux kernels by default, Windows need special drivers).

Other tuning must be perfomed on hypervisor OS level but as I reviewed GNS3VM all most obvious staff is already setup. I was looking for staff that would speed up my vSRXes and using Virtio-BLK disks gave me the most. Juniper suggests some optimalizations for nested KVM envinronment but those were already present. I am going to investigate this more.

See attached PNG, these are settings for my GNS3VM running in Lab. Actually I decreases resources but it was running stable 6 students workshop (2x vSRX, 1x Ubuntu-Desktop, 1x VPCs, 2 switches plus Cloud per lab).

Regards J.

----- Oryginalna wiadomość -----

Od: "Mathieu BEDOS" notifications@github.com Do: "GNS3/gns3-gui" gns3-gui@noreply.github.com DW: "Jarosław Bazydło" jaroslaw.bazydlo@comp.com.pl, "Mention" mention@noreply.github.com Wysłane: sobota, 7 wrzesień 2019 14:06:17 Temat: Re: [GNS3/gns3-gui] GNS3 VM for KVM hypervisor based (#2818)

The GNS3 VM for KVM hypervisor will be released with our next GNS3 version 2.2 release.

Can you add with the disk the "best config" for kvm ?

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/GNS3/gns3-gui/issues/2818#issuecomment-529102417

grossmj commented 5 years ago

I will include this script to manually start the GNS3 VM. I would be happy to provide a libvirt xml file or anything else.

#!/bin/bash

if [[ ! $(ip link show virbr0) ]]
then
  sudo apt update
  sudo apt install -y libvirt-bin
fi

if [[ ! $(ip link show tap-gns3vm) ]]
then
  echo "Creating TAP interface"
  sudo ip tuntap add dev tap-gns3vm mode tap user $(whoami)
  sudo ip link set tap-gns3vm up
  sudo brctl addif virbr0 tap-gns3vm
fi

qemu-system-x86_64 -name "GNS3 VM" -m 2048M -cpu host -enable-kvm -machine smm=off -boot order=c \
-drive file="GNS3 VM-disk001.qcow2",if=virtio,index=0,media=disk \
-drive file="GNS3 VM-disk002.qcow2",if=virtio,index=1,media=disk \
-device virtio-net-pci,netdev=nic0 -netdev tap,id=nic0,ifname=tap-gns3vm,script=no,downscript=no
jaroslaw-bazydlo commented 5 years ago

Wrote small FAQ how to create GNS3VM with virt-install in libvirt, how knows maybe someone find this usefull :-):

  1. Download GNS3VM 2.2rc4.

    $ wget https://github.com/GNS3/gns3-gui/releases/download/v2.2.0rc4/GNS3.VM.VirtualBox.2.2.0rc4.zip

  2. Unzip and unpack GNS3VM for VirtualBox

    $ mkdir GNS3.VM $ cd GNS3.VM $ unzip -x ../GNS3.VM.VirtualBox.2.2.0rc4.zip $ tar xvf GNS3\ VM.ova

  3. Convert VMDK disk to QCOW2 and place them in KVM default image directory (set propper permissions).

    $ qemu-img convert -f vmdk -O qcow2 GNS3\ VM-disk001.vmdk GNS3\ VM-disk001.qcow2 $ qemu-img convert -f vmdk -O qcow2 GNS3\ VM-disk002.vmdk GNS3\ VM-disk002.qcow2 $ sudo cp *.qcow2 /var/lib/libvirt/images $ sudo chown libvirt-qemu:kvm /var/lib/libvirt/images/'GNS3 VM-disk001.qcow2' /var/lib/libvirt/images/'GNS3 VM-disk001.qcow2'

  4. Create KVM GNS3VM guests (will be automatically started).

    $ sudo virt-install --import --virt-type kvm --name gnsv3m-2.2dev --memory 2048 --cpu host --os-variant ubuntu18.04 --disk path=/var/lib/libvirt/images/'GNS3 VM-disk001.qcow2',bus=virtio --disk path=/var/lib/libvirt/images/'GNS3 VM-disk002.qcow2',bus=virtio --network bridge=virbr0,model=virtio --graphics spice,listen=0.0.0.0 --noautoconsole

    You must use SPICE client to open console (virt-manager, virt-viewer, Remmina with SPICE plugin installed).

  5. VM guest will not boot properly since /opt could not be mounted (we used Virtio bus so disknames are vdX1 not sdX1) but allow to use maintenance shell.

    Edit /etc/fstab and change /dev/sdb1 to /dev/vdb1 for /opt filesystem line (ideally use blockid to find out /opt UUID and use instead).

    $ nano /etc/fstab

  6. Reboot VM.

    reboot

  7. Due to dialog bug in 2.2rc4 you need to execute from shell since it is missing:

    $ sudo apt update $ sudo apt install dialog

  8. Usefull operations:

    $ virsh list --all $ virsh domdisplay gnsv3m-2.2dev $ virsh dumpxml gnsv3m-2.2dev > gns2vm-virtio.xml

  9. Force to stop VM and delete it from libvort repo.

    $ virsh destroy gnsv3m-2.2dev $ virsh undefine gns3vm-2.2dev

Regards J.

----- Oryginalna wiadomość ----- Od: "Jeremy Grossmann" notifications@github.com Do: "GNS3/gns3-gui" gns3-gui@noreply.github.com DW: "Jarosław Bazydło" jaroslaw.bazydlo@comp.com.pl, "Mention" mention@noreply.github.com Wysłane: niedziela, 8 wrzesień 2019 10:47:20 Temat: Re: [GNS3/gns3-gui] GNS3 VM for KVM hypervisor based (#2818)

I will include this script to manually start the GNS3 VM. I would be happy to provide a libvirt xml file or anything else.

#!/bin/bash

if [[ ! $(ip link show virbr0) ]]
then
  sudo apt update
  sudo apt install -y libvirt-bin
fi

if [[ ! $(ip link show tap-gns3vm) ]]
then
  echo "Creating TAP interface"
  sudo ip tuntap add dev tap-gns3vm mode tap user $(whoami)
  sudo ip link set tap-gns3vm up
  sudo brctl addif virbr0 tap-gns3vm
fi

qemu-system-x86_64 -name "GNS3 VM" -m 2048M -cpu host -enable-kvm -machine smm=off -boot order=c \
-drive file="GNS3 VM-disk001.qcow2",if=virtio,index=0,media=disk \
-drive file="GNS3 VM-disk002.qcow2",if=virtio,index=1,media=disk \
-device virtio-net-pci,netdev=nic0 -netdev tap,id=nic0,ifname=tap-gns3vm,script=no,downscript=no

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/GNS3/gns3-gui/issues/2818#issuecomment-529183126

grossmj commented 5 years ago

For info, we have released the GNS3 VM for KVM: https://github.com/GNS3/gns3-gui/releases/download/v2.2.0rc5/GNS3.VM.KVM.2.2.0rc5.zip

mabed-fr commented 5 years ago

You is the best ! Congratulations !

jaroslaw-bazydlo commented 5 years ago

Works like a charm under my QEMU/KVM, thanks a lot!

Regards J.

----- Oryginalna wiadomość ----- Od: "Jeremy Grossmann" notifications@github.com Do: "GNS3/gns3-gui" gns3-gui@noreply.github.com DW: "Jarosław Bazydło" jaroslaw.bazydlo@comp.com.pl, "Mention" mention@noreply.github.com Wysłane: poniedziałek, 9 wrzesień 2019 15:48:27 Temat: Re: [GNS3/gns3-gui] GNS3 VM for KVM hypervisor based (#2818)

For info, we have released the GNS3 VM for KVM: https://github.com/GNS3/gns3-gui/releases/download/v2.2.0rc5/GNS3.VM.KVM.2.2.0rc5.zip

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/GNS3/gns3-gui/issues/2818#issuecomment-529483253

PranavBhattarai commented 2 years ago

I got recently introduced to KVM + Qemu + Virt-Manager + Libvert. It's amazing how fast and cool this open-source project is. I immediately wiped VirtualBox and planing to wipe VMware altogether.

Thank you, everyone.

marek22k commented 1 month ago

The GNS3 VM for KVM hypervisor will be released with our next GNS3 version 2.2 release.

I cannot find the image for the current version. Where is it?

grossmj commented 1 month ago

@marek22k the GNS3 VM for KVM is available there: https://github.com/GNS3/gns3-gui/releases/tag/v2.2.49

PranavBhattarai commented 1 month ago

@grossmj There should be at least a blog on "How to" using the new KVM images. In the past, after reading a lot of unofficial posts/comments, l managed to convert the VMware image into QCow2 image. It wasn't very clear, on what to do with 2 QCow2 images.

If only it was as easy as it was with how the images open in VirtualBox or in VMWare; I wish it was easy for VirtManager too.

The "start-gns3vm.sh" script will launch VirtManager, but it won't allow VirtManager to save the profile/virtual-machine-configuration. This would have made things a lot easier.

Maybe a blog or a modification in the script is helpful in terms of VirtManager for saving the session for later usecase.