Increase image disk size?

[aeikum]

My build process takes a lot of disk space, and it's running up against the limit of the current 32G disk images. Would you be open to increasing them to 64 or even larger disk sizes?

[ladar]

Yes I would. Unfortunately the configs need to fill all the unused space with 0 so the disk image can be trimmed, before getting compressed. And writing all those zeros takes quite a bit of time, depending on the hypervisor, and/or whether an SSD is being used. Since doing a full build already takes 2-4 days, I'm hesitant to add any more overhead. Which is why I've kept the disks at 32gb.

But I feel your pain. Magma and Lineage box variants have larger disks because for the same reason (builds won't fit within the 32gb limit). A short term workaround would be adding a second disk via your Vagrantfile and doing your build on that new disk/partition.

I'm hoping to get blade servers setup, which will be dedicated to the Robox project. That should speed up the build process. When that happens the overhead will be less of a concern, since I'll be able to split builds across more servers, and all the build bots should have SSDs ... (some of the existing bots use old fashioned magnetic disks, and/or SSDs which lack TRIM support), which means they are actually writing ~20gb of zeros.

Hopefully this will happen next month. I already have the blades. They just lack memory, and drives. And of course need to be setup once the parts are added.

Once I have the servers, the question will be, how big should we make we make the drives? 64gb is the easy answer. If anyone wants it larger, now would be a good time to make your case.

[aeikum]

Fair enough, thanks for the response.

For what it's worth, my build (for https://github.com/ValveSoftware/Proton/) uses about 13GB of the disk for the initial setup, and each build of Proton takes about 12GB. So a 32GB allows for one build to exist on disk; 64 would allow for about 3 total. Still kind of tight, but probably sufficient. A 128GB disk would be plenty for any use of our build system that I can imagine.

In the meantime, I wrote a little script to resize the disk image automatically. It works for debian9 and debian10 and probably most other Linux boxes. It can be found here: https://github.com/aeikum/Proton/blob/proton_4.11-localmingw/resize-vagrant-disk.sh

[mloskot]

@ladar

Once I have the servers, the question will be, how big should we make we make the drives? 64gb is the easy answer. If anyone wants it larger, now would be a good time to make your case.

I use the generic boxes to create Hyper-V VMs for some complex processing of largish geospatial data sets. For example, running https://github.com/openmaptiles/openmaptiles building MBTiles for extracts of OpenStreetMap for big countries can take 64 - 100 GB of disk space. An exceptional case of the whole planet takes 250 GB.

As a workaround, I manually resize disk using GParted Live ISO after initial provision.

@aeikum

I wrote a little script to resize the disk image automatically. https://github.com/aeikum/Proton/blob/proton_4.11-localmingw/resize-vagrant-disk.sh

#seems to be necessary for resize2fs to pick up on the new partition size
#maybe there's a way to do that without rebooting?
vagrant halt
vagrant up

Possibly, sudo partprobe may do the job instead of the reboot.

[matthew-gill]

hey. Just wondering if there is still a plan to increase the disk size from 32gb ? A couple of members of our team are at the limit, and parallels has very basic support for adding/mounting additional disks.

Any help you could give would be really appreciated (FYI, I'm using generic/ubuntu2004)

[aeikum]

@matthew-gill Can't speak for the maintainer, but I have had some luck with the vagrant disksize plugin alongside these lines in my Vagrantfile, for debian10 but probably would work on ubuntu. It resizes the disk and then resizes the partitions on provision.

Vagrant.configure(2) do |config|
  config.vagrant.plugins = ["vagrant-disksize"]
  config.disksize.size = "128GB"

...

  config.vm.define "debian10", primary: true do |debian10|
    debian10.vm.provision "shell", privileged: "true", inline: <<-SHELL
      apt-get install -y parted

      #resize the partition
      sfdisk -d /dev/sda > /tmp/sda.sfdisk

      #remove start and size params from sda3 line, sfdisk will automatically use the rest of the disk
      sed -e 's/^.dev.sda3.*$/\\/dev\\/sda3 : type=83/' < /tmp/sda.sfdisk | sfdisk --force /dev/sda

      partprobe

      #resize2fs automatically uses the whole partition
      resize2fs /dev/sda3

...

    SHELL
  end
end

[matthew-gill]

thanks @aeikum on parallels it has this weird concept called a link clone, which has thwarted a lot of my attempts for stuff like this. I'll try what you've got above though thanks :)

[briancain]

@aeikum @matthew-gill the disk size plugin only works for VirtualBox, by the way. It won't work on parallels.

[briancain]

@aeikum - By the way, if you are using VirtualBox, vagrant now has an experimental feature for a fully managed way to configure disks. You can expand the primary disk, attach new diskss, etc with Vagrant. There's also Hyper-V support, and VMWare support is coming soon.

[matthew-gill]

Ahh ok no worries. I think I've worked out a way to manually mount a disk to the directory we keep filling up (/var/lib/docker/volumes)

[ladar]

Hi folks, I brought new build bots online last week, and I've been working out all the configuration kinks with the new servers. With the additional servers, I was able to reduce the build time from 3 days, to around 20 hours for the 3.0.18 release. (Plus another 12 hours to upload all the images to the Vagrant Cloud).

I'm hoping to do another build soon 3.0.20 with 128gb disks, as an experiment. I don't know whether the change will be permanent yet, as I'm still negotiations with my colo/transit providers (VPLS/Cogent), for the space/power/bandwidth, but I wanted to keep everyone updated.

P.S. Note I'm only build VMWare, VirtualBox, libvirt, Docker, and Hyper-V at the data center right now. So until the 2011 Macbook Air I use for the Parallels boxes gets upgraded, they will stick with the smaller disk size...

[matthew-gill]

Thanks for the update @ladar. Please, if you remember, can you send me a prod if/when you do end up doing the parallels box? As parallels vs virtualbox on osx is chalk and cheese performance-wise for us! :)

[aeikum]

@ladar Using the libvirt images here, and the increased size has been working great. Thank you.