Create a disk but not add it on a standard storage bus?

[brianjmurrell]

Is there any way to use the vagrant-libvirt syntax to create a disk but not plumb it to an existing storage bus?

Specifically, I want to create an NVMe disk and plumb it with:

lv.qemuargs :value => "-drive"
lv.qemuargs :value => "format=raw,file=" + ENV['HOME'] + "/.local/share/libvirt/images/nvme_disk1.img,if=none,id=NVME1"
lv.qemuargs :value => "-device"
lv.qemuargs :value => "nvme,drive=NVME1,serial=nvme-1"

I would of course want to abstract the path to the disk in that lv.qemuargs :value to whatever the path is to the local storage pool.

[electrofelix]

Likely it would require some custom code and usage of the up trigger in vagrant.

It appears that if support was added to be able to specify the disk type as opposed to the driver type it would be possible to have the template generate the correct output. Currently we're using type to describe disk driver type rather than the disk type. https://github.com/vagrant-libvirt/vagrant-libvirt#additional-disks. So there would be a couple of changes required both in supporting a configuration option and adjusting the template.

One other helpful change might be to allow the template for disks to support skipping disks with a particular attribute https://github.com/vagrant-libvirt/vagrant-libvirt/blob/master/lib/vagrant-libvirt/templates/domain.xml.erb#L129

It's not quite clear to me if it's possible to do exactly what you want with libvirt as the docs suggest it only allows for specifying pci addresses which suggests passthru is the only mode supported. So it might be that we're partially blocked from implementing the proper support, which leaves my initial thought of some custom classes.

What I'm thinking of is if you define a custom class in your Vagrantfile that you can pass an instance of this class to the trigger using a ruby block https://www.vagrantup.com/docs/triggers/configuration#ruby to handle creating the file. This should be called after configuration is evaluated as it gets 2 options env and machine. These would allow accessing the libvirt provider driver connection via env[:machine].provider.driver.connection and support locating the correct storage pool.

You should be able to include the functionality of https://github.com/vagrant-libvirt/vagrant-libvirt/blob/master/lib/vagrant-libvirt/util/storage_util.rb by simply relying on vagrant to have made sure the plugin is loaded and add the include line to your class as done by https://github.com/vagrant-libvirt/vagrant-libvirt/blob/929654b83aec63f3e7c68101d3b6582ac32febe6/lib/vagrant-libvirt/action/create_domain.rb#L8

That should allow use of some helpers.

https://github.com/vagrant-libvirt/vagrant-libvirt/blob/929654b83aec63f3e7c68101d3b6582ac32febe6/lib/vagrant-libvirt/action/create_domain.rb#L158 shows how that can be used and I think in your case, just need to determine what path you want (or use a helper to construct one as done for https://github.com/vagrant-libvirt/vagrant-libvirt/blob/929654b83aec63f3e7c68101d3b6582ac32febe6/lib/vagrant-libvirt/action/create_domain.rb#L162) and create the disk as shown at https://github.com/vagrant-libvirt/vagrant-libvirt/blob/929654b83aec63f3e7c68101d3b6582ac32febe6/lib/vagrant-libvirt/action/create_domain.rb#L184-L206

In fact this does suggest there might be benefits in moving some of the disk creation code to helpers so it can be called directly with arguments instead of being inline.

What you can do is store the information about the disk absolute paths using a ruby class variable (use prefix @@) so that you can pass a reference to the class or a helper class directly to the lv.qemuargs setting:

lv.qemuargs :value => "-drive"
lv.qemuargs :value => MyStorageHelperClass.new
lv.qemuargs :value => "-device"
lv.qemuargs :value => "nvme,drive=NVME1,serial=nvme-1"

Make sure the class contains a method to_s and have that return:

def to_s
    return "format=raw,file=#{@@disk_path},if=none,id=NVME1"
end

You can also pass the disk name as an argument to the class and have it use the same helpers to resolve the path. If you can use the helpers here, could create the class instance with the disk name and have it perform the same delayed resolving that is done in the up trigger ruby block without needing to use the class instance variable at all. The main reason for the delayed resolving though is to ensure the provider config has been resolved to allow creation of the driver connection and resolve the storage pool correctly.

Unfortunately I don't see any easier way to do this for now if you want to be able to abstract the resolving of the path. It might be more straight forward to start with a hardcoded path and create the disk with the up trigger and assume a specific known path.

Hopefully this provides a couple of options to allow you to consider which direction to head.

[brianjmurrell]

While all of that is really awesome help, I'm in a mode of finding the shortest path between two points here. Maybe one day I can spend more time and be more adventurous about trying to implement some of your suggestions above, I ended up with:

        (1..3).each do |ss_idx|
                config.vm.define "vm#{ss_idx}", autostart: true do |ss|
                        ss.trigger.before :up do |trigger|
                                trigger.ruby do |env,machine|
                                        (1..2).each do |nvme_idx|
                                                if File.exist?(ENV['HOME'] + "/.local/share/libvirt/images/nvme_disk#{ss_idx}-#{nvme_idx}.img")
                                                        File.delete(ENV['HOME'] + "/.local/share/libvirt/images/nvme_disk#{ss_idx}-#{nvme_idx}.img")
                                                end
                                                system("qemu-img create -f raw " + ENV['HOME'] + "/.local/share/libvirt/images/nvme_disk#{ss_idx}-#{nvme_idx}.img 32G")
                                                system("restorecon " + ENV['HOME'] + "/.local/share/libvirt/images/nvme_disk#{ss_idx}-#{nvme_idx}.img")
                                        end
                                end
                        end
                        ss.vm.provider :libvirt do |lv|
                                # An NVMe drive:
                                (1..2).each do |nvme_idx|
                                        lv.qemuargs :value => "-drive"
                                        lv.qemuargs :value => "format=raw,file=" + ENV['HOME'] + "/.local/share/libvirt/images/nvme_disk#{ss_idx}-#{nvme_idx}.img,if=none,id=NVME#{ss_idx}-#{nvme_idx}"
                                        lv.qemuargs :value => "-device"
                                        lv.qemuargs :value => "nvme,drive=NVME#{ss_idx}-#{nvme_idx},serial=nvme-1234#{ss_idx}#{nvme_idx}"
                                end
                        end

Having a hard-coded path to the libvirt storage location in there is nasty, to be sure. I'm not sure if vagrant-libvirt->Vangrantfile knows about it to be able to abstract that out or not.

[electrofelix]

@brianjmurrell makes total sense to start with the hardcoded approach, if I get some time to mess around I might try to create an example, I think it can be useful to show some of the more advanced tricks for scenarios like this where support doesn't exist yet. Essentially that was one of the reasons behind supporting qemuargs and qemuenv

[brianjmurrell]

Yeah, the ability to do ad-hoc configurations with qemuargs and qemuenv is awesome!