Option to deploy HD files on creation instead of on start.

[spikefishjohn]

Is your feature request related to a problem? Please describe. I've noticed that GNS3 doesn't create the images files for, at least qemu imges until you start the VM. I've tested this with 2.2.7 with the GUI and API. The issue I have is I want to create the image, then use guest fish to modify the image before starting the image. As it stands we have to issue a start followed by a stop right after that to get the files for the device to deploy.

Describe the solution you'd like I just started looking through code trying to figure out when qemu drives are deployed, but I'm assuming there is a reason for delay until start so maybe having an API option for create to create files right now? Not sure how hard that is.

Describe alternatives you've considered quick start / stop to get files to deploy.

Additional context

[spikefishjohn]

Wanted to bring this up again. I've been playing around with automation and it takes much longer to create devices (really qemu) one by one so they can be started/stopped quickly. I started looking through the code and i'm not understanding what is triggering deploying of files. Any chance I can get a pointer so I can take a stab at coming up with a patch?

[spikefishjohn]

@grossmj bump

[josephmhiggins]

oh...this is interesting and related to something else I am doing. Using the "builder" pattern to prepopulate a qemu hard disks before you start a VM can "drastically" reduce the size of an exportable project.

For an Ubuntu 22 VM, it reduces from about 400MB to 288MB (I justed tested it 8 hours ago) I would say any coding change ......scratch that....well if spikefishjohn wants to do it ....that looks like a lot of work to me.

I do not know what, if any, downside there is to prepopulate a qemu hard disk. There may be a downside to such a project, but I do not know what the downside is. It looks kinda cool project.

Edit: sorry, an empty Ubuntu qcow2 is essentially 0MB and then thats "the .gns3 file". If I start it, the file is 288MB. If I login via VNC ubuntu org and put a ip address on an interface, it is about 400MB. Your idea is interesting spike, but I believe it is a waste of time for my scenario. I would just create a .gns3 file and a script and send that instead of a portable project file. But, if you have a use case, then you can create your own path.

Edit: extremely sorry. but there would have to be some RFC QEMU standard signal called "everything okay you are ready for the first login". I do not know if that makes sense. It is a cool idea though.

[josephmhiggins]

I am highly convinced what you want to do in code is completely impossible.

The standard technique, nowadays, is to create an overlay image from an original OEM vendor's VM.

So, as an example with Ubuntu 1) download Ubuntu 22 from osboxes.org and install it in GNS3 2) copy that original image, 'Ubuntu 22.04 (64bit).vmdk', to something like, Ubu-22-04-iperf-ssh-master.vmdk 3) install Ubu-22-04-iperf-ssh-master.vmdk as a custom image in GNS3 4) in the gns3 Advanced settings of the qemu image of Ubu-22-04-iperf-ssh-master.vmdk, unclick Use as Linked Based VM, 5) drag that image into the gns3 topology of a new project called something like Ubu-22-iperf-ssh in a directory something like your_hopepath/GNS3/projects/datastore or your_hopepath/GNS3/projects/prebuilt 6) install iperf and ssh server on the vm 7) close gns3, start gns3, and go immediately to advanced settings of the Ubu-22-04-iperf-ssh-master reclick Use as linked based VM 8) point your automation scripts to that Ubu-22-04-iperf-ssh-master.vmdk because it is prebuilt.

It is a standard technique.

For an academic viewpoint of this technique, you can try this website but websites can disappear over time: https://kashyapc.fedorapeople.org/virt/lc-2012/snapshots-handout.html

As far as an alternate method of building only an iso image to use as a "configuration helper", I see no time savings this way.

Cisco may have a special tool they built only for their IOS-XE. Someone in the GNS3 community knows how to do this. But I forget who. The poster posted it about 7 months ago. You can try to post on GNS3 about the IOS-XE and maybe the original poster will show up. The poster is really good.

[josephmhiggins]

Impossible is defined as taking a ridiculous amount to complete the task.

So...as an example, you have would have to manually get the "everything okay you are ready for the first login" string via VNC or telnet or maybe aux also, for every single VM manufactured by every single OEM vendeor and then trigger some action based on that string.

Impossible.

[spikefishjohn]

I'm thinking you might have missed what the bug is. Let me show you.

I just made a new project and added a single debian 11.4 image. I right clicked on it and click show in file manager to get this message.

As you can see, even though the VM has been deployed to the project map the directory where the VM will live has not been created.

Now I start the VM

Now I stop the VM

And now the HD images have been deployed.

This bug report is to request a way for GNS3 to deploy the VM without me starting and stopped to deploy. Currently I'm starting and stopping a VM before editing the HD image with guest fish. I'm currently deploying network config and uploading files I want the VM to start with. Everything already works, this bug report would just make it a little bit faster to deploy.

BTW why would you use a vmdk over a qcow2 image?

[grossmj]

I guess we could add an API param when we call the create node API endpoint. Do you need this to be in the GUI as well?

[spikefishjohn]

I don't think its needed at the gui level.

Question, what was the reason to wait until initial start of a VM to deploy the HD images? I mean, my knee jerk reaction is deploy images when dropped on the map, but since I don't know why this was chosen i'm assuming i'm missing something.

An API bool option seems like it would be fine. earlydeploy = True / False

something like that?

[josephmhiggins]

@spikefishjohn Yeah, strictly speaking, I can not see any reason to not earlydeploy it. On the other hand, it only saves about 1/100,000 of a second. One case in point, the cisco ise take 90 minutes to boot for the first time after I start it.

Another case in point, if you are studying BGP-EVPN and you are uploading a lab to the gns3 marketplace and you are using Cisco-NXOSv in your lab, then I would want you to boot that guy up all the way to the login and I think also give it the NX-OSv a hostname and save the config. The reason why I would want you to do that is to avoid me getting dropped into bootloader mode and then having to delete the NX-OSv from the topology and dragging a new one back in and maybe having to reconnect 32 links in the gns3 topology ughh that would be a nightmare. I would probably delete the lab and reimport it.

[spikefishjohn]

@josephmhiggins I'm not following what you're getting out, but I think maybe you're not getting what the bug report is about. This is only to cover when a HD image is put into the folder where the VM lives. As of right now that only happens the first time you hit start. This report is to make it happen when you put it on the map or to at least give the API an option to make that happen.

This bug report doesn't cover anything outside of that.

[grossmj]

Question, what was the reason to wait until initial start of a VM to deploy the HD images? I mean, my knee jerk reaction is deploy images when dropped on the map, but since I don't know why this was chosen i'm assuming i'm missing something.

This has been implemented a long time ago but I think the reason we create the HD images on start is because we want to be sure we can create the nodes when loading a topology. A node would be in an inconsistent state if for whatever reason there is an error during its creation. After saying that, I realize we could ignore any error during the creation and try again during the start...

Another possible reason is we just wanted to give a chance to users to add more HD images after a node is created and all the HD images are deployed once the node is started.

An API bool option seems like it would be fine. earlydeploy = True / False

Yes, this is what I have in mind :)

[spikefishjohn]

Just dawned on me that by not auto deploying it would in theory give you a window to link or bind mount the vm directory before the images deploy.

Btw I’ve never used this before but in theory if someone wanted to do this with the gui could they add a key / value through the advanced section of the gui? I’m not in a place where I can look at the moment but hopefully you know what I’m talking about.