Enhancement: Support automatic VDI compaction based on Guest TRIM/UNMAP/Discard commands

[ghost]

Background

A common issue with thin proivisioned (a.k.a. dynamic) virtual disk files is that they grow beyond the size of the actual Guest data. This is because when a guest is deleting data from the filesystem, the files are only unlinked. Therefore the old data remains in the VDI.

With the new SSD drives OS'es now have the possibility to actually tell the storage layer what blocks are no longer in use through the ATA TRIM or the SCSI UNMAP command sets.

Automatic compaction of VDI files

Using the TRIM support in the guest OS it should be possible to automatically compact the VDI files used in Xen, by intercepting the TRIM or UNMAP commands.

• This is possible in VirtualBox by emulating an SSD drive in the VM settings. nonrotational="true" discard="true

      <AttachedDevice nonrotational="true" discard="true" type="HardDisk" hotpluggable="false" port="1" device="0">
        <Image uuid="{a8763005-3f43-4ba8-843a-eedae46989ff}"/>
      </AttachedDevice>

• It is also possible in VMWare, though I believe they use a guest driver or tool instead of TRIM.
• Proxmox does this with TRIM and QCOW2 files: https://pve.proxmox.com/wiki/Shrink_Qcow2_Disk_Files

Of course the huge advantage is to be able to keep thin provisioned VDI storage thin with a low-resource automatic compaction. Today we have to fill a disk with zeroes and then either turn the VHD files sparse or use some VHD magic to compact disks. Very time consuming and not user or admin friendly.

Looking at the schematics at https://xapi-project.github.io/xapi-storage/#learn-architecture it may be possible to do this with a volume plugin. Possibly the Xapi needs to be expanded to forward the TRIM request to the plugin.

[MarkSymsCtx]

This project defines the control plane API to the storage layer from the toolstack. There is no implementation of any of the APIs within this project. Implementations of the API may use whatever method they like to handle the storage, indeed some already use qcow2 and will where supported by the underlying storage see that the guest initiated TRIM operations result in space reclamation.

[ghost]

OK. I was misinformed. Could you perhaps refer me to the right project for a request like this?

[MarkSymsCtx]

There isn't one, there are no public, open source, implementations of the storage APIs. The implementations of the API undertaken by Citrix are proprietary.