search

WhoopInc / vagrant-s3auth

Vagrant plugin for private, versioned boxes on Amazon S3.
MIT License
108 stars 23 forks source link

Invalid URL translation. #34

Open macdabby opened 6 years ago

macdabby commented 6 years ago

I just started seeing this when updating to vagrant 2.0.1 and virtualbox 5.2.x, however I still see it now when I downgrade to vagrant 1.9.1 and virtualbox 5.1.x

vagrant-s3auth is v1.3.2

In the vagrant file I have defined

config.vm.box = "boxnamae"
config.vm.box_url = "s3://path/to/box"
REGION = "us-east-1"

I get the following output:

==> default: Checking if box 'boxname' is up to date...
The metadata for the box was malformed. The exact error
is shown below. Please contact the maintainer of the box so
that this issue can be fixed.

751: unexpected token at '<html><head><meta http-equiv="refresh" content="0;url=http://lookup.t-mobile.com/index.php?origURL=http://s3-placeholder.amazonaws.com/path/to/box&bc="/></head><body><script type="text/javascript">window.location="http://lookup.t-mobile.com/index.php?origURL="+escape(window.location)+"&r="+escape(document.referrer)+"&bc=";</script></body></html>'

My ISP is tmobile and this is their default response for requesting a page where the DNS can't be found. Notice that it's trying to access http://s3-placeholder.amazonaws.com/path/to/box with the domain s3-placeholder.... I believe that should be something like us-east-1.amazonaws.com or something like that. I looked in the source for this plugin but i wasn't able to understand where it replaces this placeholder.

benesch commented 6 years ago

That placeholder URL will get picked up by the S3_HOST_MATCHER: https://github.com/WhoopInc/vagrant-s3auth/blob/fb112e99b4e9d4228ffc1d83c02ee0f3c79ffe20/lib/vagrant-s3auth/util.rb#L47-L51

At least, it will in theory. I'll look into this.

macdabby commented 6 years ago

Interesting, I was just able to test this on another network. On Network A (tmobile) nonexistingdomain.com returns a DNS entry for Network A's search/notfound page with HTTP code 200. On network B, DNS returns no entries. On network B, I do not get the error mentioned, and the box starts up just fine.

So it seems like maybe this is what's happening: 1) Vagrant attempts to load http://s3-placeholder.amazonaws.com/path/to/box 2) Only if that fails, the URL is reprocessed in the S3_HOST_MATCHER mentioned above to the correct URL. 3) If the s3-placeholder.amazonaws.com URL returns a 200 with content, vagrant attempts to parse this content as the expected response.

Additional note: This had nothing to do with the upgrade, that was just coincidence.

benesch commented 6 years ago

Ah, yes, that theory sounds exactly right! vagrant-s3auth tries to be a good citizen by letting Vagrant attempt to download the box first. If and only if the download of that box fails does vagrant-s3auth kick in. We look for failures at the HTTP layer, though, so TMobile's garbage looks like valid box metadata until a higher layer of Vagrant actually tries to parse it as JSON.

I'm struggling to remember why I switched from "http://s3.amazonaws.com" to "http://s3-placeholder.amazonaws.com", and the commit message, er, is less than helpful (https://github.com/WhoopInc/vagrant-s3auth/commit/ee6086bce3cebc74ab1d0da361d8ec9cacbe6e8e). I'll have some time tomorrow to dig in.

paul-court commented 6 years ago

+1

This also happens for virgin media customers in the UK. By default the network turns on one of those annoying search results pages. So http://s3-placeholder.amazonaws.com does indeed resolve. :-/

The final result is that a chunk of html gets downloaded and vagrant fails to untar it with a message like this:-

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'propertyfile/standard_1404' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Box file was not detected as metadata. Adding it directly...
==> default: Adding box 'propertyfile/standard_1404' (v0) for provider: virtualbox
    default: Downloading: s3://1psg-infrastructure/PropertyFile/vagrant/1404/standard.json
    default: Signing S3 request with key '<REDACTED>' loaded from profile '1psgdev'
The box failed to unpackage properly. Please verify that the box
file you're trying to add is not corrupted and that enough disk space
is available and then try again.
The output from attempting to unpackage (if any):

bsdtar: Error opening archive: Unrecognized archive format
boneill42 commented 6 years ago

What's the status on this one? (i'm +1 as well)

chrisemerson commented 3 years ago

I am also now seeing this, again because of the Virgin Media search page (which won't turn off despite me trying to turn it off). Any update on this please?

chrisemerson commented 3 years ago

Annoyingly it looks a lot like this project has been abandoned :/

benesch commented 3 years ago

Annoyingly it looks a lot like this project has been abandoned :/

Indeed. I haven't been employed by @WhoopInc in six years, and I doubt they're even still using this plugin. CI broke constantly due to constant Travis CI, Vagrant, and AWS changes, which were too exhausting for me to keep up with in my free time.

If you'd like to submit a PR, I'd be delighted to take a look provided you can get it to pass tests. See my comment here: https://github.com/WhoopInc/vagrant-s3auth/pull/39#issuecomment-504586063.

chrisemerson commented 3 years ago

Annoyingly it looks a lot like this project has been abandoned :/

Indeed. I haven't been employed by @WhoopInc in six years, and I doubt they're even still using this plugin. CI broke constantly due to constant Travis CI, Vagrant, and AWS changes, which were too exhausting for me to keep up with in my free time.

If you'd like to submit a PR, I'd be delighted to take a look provided you can get it to pass tests. See my comment here: #39 (comment).

I would love to but I don't know any Ruby at all unfortunately :( I've fixed this for now on Virgin Media by switching to google's DNS service.