lavabit / robox

The tools needed to robotically create/configure/provision a large number of operating systems, for a variety of hypervisors, using packer.
620 stars 140 forks source link

checksum error in vagrant box download of generic/freebsd13 #287

Closed iameru closed 8 months ago

iameru commented 8 months ago

When I spin up a box with generic/freebsd13 I get a checksum error persistently, across multiple mashines. I am not sure if this error is on your or hashicorps side.

vagrantfile

vm_name = "freebsd-vm"

Vagrant.configure("2") do |config|
  config.vm.define vm_name
  config.vm.hostname = vm_name
  config.vm.box = "generic/freebsd13"
end

vagrant up

Bringing machine 'freebsd-vm' up with 'libvirt' provider...
==> freebsd-vm: Box 'generic/freebsd13' could not be found. Attempting to find and install...
    freebsd-vm: Box Provider: libvirt
    freebsd-vm: Box Version: >= 0
==> freebsd-vm: Loading metadata for box 'generic/freebsd13'
    freebsd-vm: URL: https://vagrantcloud.com/generic/freebsd13
==> freebsd-vm: Adding box 'generic/freebsd13' (v4.3.4) for provider: libvirt
    freebsd-vm: Downloading: https://vagrantcloud.com/generic/boxes/freebsd13/versions/4.3.4/providers/libvirt/amd64/vagrant.box
    freebsd-vm: Calculating and comparing box checksum...
The checksum of the downloaded box did not match the expected
value. Please verify that you have the proper URL setup and that
you're downloading the proper file.

Expected: ef99c96f929bc701f30cb113e1f5a1dee43f4f4d8ec0ec318a7ec2d02e5059e8
Received: d5442a6f1a84db8e787bbcfef53418146123ca4671ad674144d027fa87840577

on another computer where I have a working generic/freebsd I can spin up mashines, but when I update the box I get the exact same error.

vagrant box update

==> freebiggydiggy: Checking for updates to 'generic/freebsd13'
    freebiggydiggy: Latest installed version: 4.2.16
    freebiggydiggy: Version constraints:
    freebiggydiggy: Provider: libvirt
==> freebiggydiggy: Updating 'generic/freebsd13' with provider 'libvirt' from version
==> freebiggydiggy: '4.2.16' to '4.3.4'...
==> freebiggydiggy: Loading metadata for box 'https://vagrantcloud.com/generic/freebsd13'
==> freebiggydiggy: Adding box 'generic/freebsd13' (v4.3.4) for provider: libvirt
    freebiggydiggy: Downloading: https://vagrantcloud.com/generic/boxes/freebsd13/versions/4.3.4/providers/libvirt/amd64/vagrant.box
    freebiggydiggy: Calculating and comparing box checksum...
The checksum of the downloaded box did not match the expected
value. Please verify that you have the proper URL setup and that
you're downloading the proper file.

Expected: ef99c96f929bc701f30cb113e1f5a1dee43f4f4d8ec0ec318a7ec2d02e5059e8
Received: d5442a6f1a84db8e787bbcfef53418146123ca4671ad674144d027fa87840577

Am helping myself now via

vagrant box add --box-version=4.2.16 generic/freebsd13 --provider=libvirt

4.2.16 is working

ladar commented 8 months ago

@tameru THANK YOU for letting me know. This does happen on occasion. I was the person who pushed hard to get HashiCorp to add support for the embedded hash, precisely because of the corruption. I was hoping it would be a stopgap on the way to adding support for cryptographic signatures but I digress.

The problem can occur during download, in which case redownlading fixes it. Occasionally the uploads also get corrupted, which is what happened here. The root of the issue is that Vagrant cloud doesn't confirm whether the uploaded file matches the hash. Which is what happened here.

I've always had issues uploading releases to the Vagrant cloud, which is why I throttle the process, which reduced the error rate, but meant it the process takes signifcantly longer. I have to use ./robox.sh public afterwards to find any uploads that failed, and then upload them manually which is a huge pain.

Anyways, I've noticed mroe problems uploading as of late. In some cases it was uploaded errors, in others it was a delay of 4-5 hours between the upload, and the file being available for download. Which is why I overhauled the script I use for uploading (the direct.sh script), when I added arch support/and modified it to use the v2 API. That process started with 4.3.4 rekeasem wgucg is probably why that box got muned. I overhauled it again with teh 4.3.6 release, because I ran into a 30% failure rate. Hopefully it gets better. But let me know if you run into more box corruption issues.

I used to download all the files after an upload to confirm they were good, but that hasn't been possible for a long time, because of the daily use limits in place. But I did check the box you mentioned, and it looks good now, so I'm closing this issue.

[robox]# res/scripts/match.sh generic freebsd13 libvirt amd64 4.3.4
Box  +  generic freebsd13 libvirt amd64 4.3.4
iameru commented 8 months ago

Thanks for your effort in improving the supply chain here. I would expect hashicorp to check for corruption, if a checksum is offered and wish for signing aswell, but as you say yourself, this is not the topic here. Thanks for your effort! If you say it is fixed it probably is, will get around trying it earliest in 2 days. thank you