Can't get metadata boxes to work properly?

[harlanbarnes]

I'm sure I'm doing something wrong, but I'm stuck. I feel positive I'm doing something stupid, but I can't spot it.

I have a json file called base-precise64.json and it is in a private space on an S3 bucket. We'll call that S3 bucket mycompany.

I'm simply trying a vagrant up --provider lxc. (I tried it with other providers as well with no luck.)

Here's my relevant Vagrant config:

Vagrant.configure('2') do |config|
  config.vm.box     = 'mycompany/base-precise64'
  config.vm.box_url = 's3://mycompany/private/base-precise64.json'
end

The base-precise64.json looks something like this:

{
  "name": "mycompany/base-precise64",
  "description": "Base Installation of Ubuntu Precise amd64",
  "versions": [
    {
      "version": "1.2.3",
      "providers": [
        {
          "name": "lxc",
          "url": "https://s3.amazonaws.com/mycompany/private/source/base-precise64-20141015T185317Z-lxc.box",
          "checksum_type": "sha256",
          "checksum": "d0be816ca323952535b6ab3fc5ff07608eb19496edd6d80af50208df28f388ef"
        },
        {
          "name": "virtualbox",
          "url": "https://s3.amazonaws.com/mycompany/private/images/source/base-precise64-20141015T185317Z-virtualbox.box",
          "checksum_type": "sha256",
          "checksum": "4bb9a0f9573781208ba6dabe52d3f3eacb2408dc80d9657a7cb5133906a5537d"
        },
        {
          "name": "aws",
          "url": "https://s3.amazonaws.com/mycompany/private/images/source/base-precise64-20141015T185317Z-aws-ebs.box",
          "checksum_type": "sha256",
          "checksum": "fc821c970925d926341d149229e024000e2ecd617f4874e2aee118f2de8d1037"
        }
      ]
    }
  ]
}

The error I am getting is:

bsdtar: Error opening archive: Unrecognized archive format

The box failed to unpackage properly. Please verify that the box
file you're trying to add is not corrupted and try again. The
output from attempting to unpackage (if any):

bsdtar: Error opening archive: Unrecognized archive format

I've turned on the VAGRANT_LOG and added some sleep 30 statements in Vagrant's execute_curl method in Vagrant::Util::Downloader to see what files are being created. (I was first concerned that the AWS creds weren't making it or some other general auth problem.)

From the logs, I can see curl being executed twice ... apparently for the same base-precise64.json file. The first time, the logs say this:

 INFO box_add: Downloading box: https://s3.amazonaws.com/mycompany/private/images/base-precise64.json => /home/hbarnes/.vagrant.d/tmp/box20d6fd9a9557f951fbee211ce1308ddcd42d9b44
 INFO downloader: HEAD: https://s3.amazonaws.com/metavault/private/images/base-precise64.json
 INFO downloader: s3auth: Signing S3 URL: https://s3.amazonaws.com/metavault/private/images/base-precise64.json
 INFO subprocess: Starting process: ["/opt/vagrant/bin/../embedded/bin/curl", ...
 INFO interface: output: Adding box 'mycompany/base-precise64' (v0) for provider: lxc
 INFO interface: output: ==> default: Adding box 'mycompany/base-precise64' (v0) for provider: lxc

but don't seem to download the file (even though it says it is). Then again, I guess a HEAD should't produce an actual download.

The second time, it produces a file like /home/hbarnes/.vagrant.d/tmp/box20d6fd9a9557f951fbee211ce1308ddcd42d9b44. The contents of this tmp file is the contents of base-precise64.json. So it's successfully authenticating against S3 to pull down the metadata file.

Here's where the breakage happens. Vagrant seems to think this is a normal box file and not a metadata file to lookup the ACTUAL box file from. And, obviously, that's why the bsdtar command fails.

As a side note, I also tried using the metadata JSON file locally and THAT seems to work fine. It figures out that it needs to download the box from the information in the file and continues on like normal.

Here's the full log: https://gist.github.com/harlanbarnes/0eb56fa5fec2c0caa140

Do you have any ideas/pointers from me to look?

Thanks for your time.

[butterflyhug]

@benesch Do you have any thoughts on this?

[waisbrot]

Do you get the same error with other providers? I've only tried Vagrant with containers once and I couldn't get it to work (independent of s3auth).

You can test that you're box is OK by downloading it from s3 yourself (via Curl), then manually adding it to vagrant with vagrant box add. Does that work?

[waisbrot]

Metadata-wise, I see that we use sha1 while you're using sha256. Probably not the issue.

When we upload the metadata file, we specify a content-type of application/json to S3. Try that; I think I recall having strange problems before because I let AWS guess the content type.

[harlanbarnes]

As soon as I saw your content-type response, I knew that was going to be it. And it was! I've been bitten by that before in various other scenarios.

Thanks so much for that (and for the plugin). That was the last glitch in the most rockin' private image distribution configuration I've ever made. I can't even describe the horrendous ideas I had to trying to make a private image repo before I found this plugin.

Anyway, using Packer, plus some Ruby glue to make the metadata boxes and this plugin has made it all work nicely.

Thanks again,

hb

[stephen-mw]

Yikes. I got bit by this. For those that stumble here, you'll need to upload your file to S3 with "application/json" as your content encoding.

It's easy to do with the awscli tools:

aws s3 put --content-encoding "application/json" local_file s3://bucket/remote_file

[benesch]

Re-opening this until I can add a note to the README documenting this.

[benesch]

Cheers!