hashicorp / vagrant

Vagrant is a tool for building and distributing development environments.
https://www.vagrantup.com
Other
26.05k stars 4.42k forks source link

use the -S flag with bsdtar for sparse extraction of boxes #13252

Closed mattlqx closed 10 months ago

mattlqx commented 11 months ago

from bsdtar's man page:

     -S      (x mode only) Extract files as sparse files.  For every block on disk, check first if it contains only NULL bytes and seek over it otherwise.  This works similar
             to the conv=sparse option of dd.

This is particularly important when dealing with MacVMs that use a raw disk image file that can't be resized after creation. A MacVM's disk0.img that it uses as its primary boot device is sparsely allocated so typically you want to size it fairly large since you don't get another opportunity to. In the current state, this unfortunately causes Vagrant box sizes to be huge and can cause space issues trying to work with a single VM since you need...

  1. temporarily the size of the compressed box + the extracted box living at ~/.vargant.d/boxes then...
  2. the size of the extracted box plus a copy of the vm image that gets imported into the virtualization app, Parallels in my case, so effectively 2x the size of the image.

If sparse images are respected here at the source, it can make things much easier down the line.

As a side note, if anyone is wondering, the disk0.img can be made sparse by have a final guest provisioning step that uses diskutil apfs resizeContainer down to near the actual usage size, which will relocate all data in the img file into that range, then can be resized back with the same command to maximum giving you a "compacted" disk0.img.

I'm not sure if there are specific reasons why this flag is not used already with bsdtar (like backward compatibility? not sure what version or date this flag was introduced) but I haven't seen any discussion on it in this repo.

hashicorp-cla commented 11 months ago

CLA assistant check
All committers have signed the CLA.

mattlqx commented 10 months ago

Awesome, thanks!