Optimize VHD to reduce image size - Githubissues

cloudbase / windows-imaging-tools

Tools to automate the creation of a Windows image for OpenStack, supporting KVM, Hyper-V, ESXi and more.

Apache License 2.0

675 stars 227 forks source link

Optimize VHD to reduce image size #265

Closed ader1990 closed 5 years ago

ader1990 commented 5 years ago

Changes:

Use Optimize-VHD to optimize the allocation of space. The Compact operation is used to optimize the files. This operation reclaims unused blocks as well as rearranges the blocks to be more efficiently packed, which reduces the size of a virtual hard disk file.

To use Optimize-VHD, the virtual hard disk must not be attached or must be attached in read-only mode.

hyper-v: remove auto-checkpoint and use static memory

Windows 10 1903 version creates automated checkpoints by default, which can add a delay in the image generation time.

Static memory is also enabled, as dynamic memory adds a penalty in the VM performance.

AppVeyorBot commented 5 years ago

:x: Build windows-openstack-imaging-tools 1.0.35 failed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/8067209aaf by @ader1990)

AppVeyorBot commented 5 years ago

:x: Build windows-openstack-imaging-tools 1.0.36 failed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/74daebd4cc by @ader1990)

AppVeyorBot commented 5 years ago

:x: Build windows-openstack-imaging-tools 1.0.37 failed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/5b8549b6f7 by @ader1990)

AppVeyorBot commented 5 years ago

:x: Build windows-openstack-imaging-tools 1.0.38 failed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/ca28a3fb48 by @ader1990)

AppVeyorBot commented 5 years ago

:white_check_mark: Build windows-openstack-imaging-tools 1.0.39 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/ca7c4fc4cd by @ader1990)

AppVeyorBot commented 5 years ago

:white_check_mark: Build windows-openstack-imaging-tools 1.0.40 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/6cbf981101 by @ader1990)

AppVeyorBot commented 5 years ago

:white_check_mark: Build windows-openstack-imaging-tools 1.0.43 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/6d5d5d5b40 by @ader1990)

AppVeyorBot commented 5 years ago

:white_check_mark: Build windows-openstack-imaging-tools 1.0.44 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/cc43853b3a by @ader1990)

AppVeyorBot commented 5 years ago

:white_check_mark: Build windows-openstack-imaging-tools 1.0.47 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/f9ba24d3ed by @ader1990)

felfert commented 5 years ago

Unfortunately, I can't test this, because I don't have Hyper-V

ilenghel commented 5 years ago

I have tested the PR and it works very well. VHDX size for Windows Server 2016 with all updates installed dropped from 19 GB to 13 GB. +1 from me.

ader1990 commented 5 years ago

Unfortunately, I can't test this, because I don't have Hyper-V

@felfert I assume that s why you got a different image size. What was your workflow?

felfert commented 5 years ago

Unfortunately, I can't test this, because I don't have Hyper-V

@felfert I assume that s why you got a different image size. What was your workflow?

Huh, I assumed you know your scripts ;-)

I'm using a modified Examples/create-windows-cloud-image.ps1 which in turn uses New-WindowsCloudImage. After that I just do what the description of New-WindowsCloudImage says:

I copy the resulting qcow2 image to my libvirt-based kvm hypervisor and run it there once. This run inflates it to 29G. After that, I convert it once more from qcow2 to qcow2 compressed (which reduces the size to 5G - IF sdelete is run during the kvm-run) and finally upload it to openstack

ader1990 commented 5 years ago

Unfortunately, I can't test this, because I don't have Hyper-V

@felfert I assume that s why you got a different image size. What was your workflow?

Huh, I assumed you know your scripts ;-)

I'm using a modified Examples/create-windows-cloud-image.ps1 which in turn uses New-WindowsCloudImage. After that I just do what the description of New-WindowsCloudImage says:

I copy the resulting qcow2 image to my libvirt-based kvm hypervisor and run it there once. This run inflates it to 29G. After that, I convert it once more from qcow2 to qcow2 compressed (which reduces the size to 5G - IF sdelete is run during the kvm-run) and finally upload it to openstack

I understand, from the initial report, it was not clear whether you ran it in PS.

In your case, it makes perfect sense to have something like sdelete in your workflow. If you had generated on Windows, dism updates cleanup + defragment + Optimize-Partition and Optimize-VHD would have done the trick. as you have only dism cleanup + defragment, you need something similar, which in your case can be sdelete.

But you can also apply a trick and do an additional qcow2 conversion, which you have already done. sdelete won't help you that much in your case, after all I know. can you check if my assumption is true? what is the difference between using sdelete + additional conversion and just an additional conversion?

Thank you, Adrian Vladu

felfert commented 5 years ago

But you can also apply a trick and do an additional qcow2 conversion, which you have already done. sdelete won't help you that much in your case, after all I know. can you check if my assumption is true? what is the difference between using sdelete + additional conversion and just an additional conversion?

Thank you, Adrian Vladu

That exactly the reason why I made this PR: Without running sdelete, the additional conversion practically no effect at all.

Well, I investigated for an OSS replacement but found nothing. (There is ntfswipe from ntfstools which is now part of ntfs-3g, however this is working on linux fuse-blocklevel and not really fits in here). So, I will write a simple open-sourced replacement myself ...