search

heroku / heroku-buildpack-ruby

Heroku's buildpack for Ruby applications.
MIT License
8 stars 1 forks source link

U.S.-based, slow download by default #359

Closed awendt closed 9 years ago

awendt commented 9 years ago

I'm in Europe and every time I try to download something from s3-external-1.amazonaws.com, it's slow as hell, up to a point where timeouts are hit:

$ curl --fail --retry 3 --retry-delay 1 --connect-timeout 5 --max-time 30 https://s3-external-1.amazonaws.com/heroku-buildpack-ruby/cedar-14/ruby-2.2.0.tgz -o ruby-2.2.0.tgz
curl: (28) Operation timed out after 29960 milliseconds with 4806624 out of 16684197 bytes received

A quick check showed this endpoint is based in Seattle, WA.

These timeouts happen from home as well as from the office (where we have a ~50 Mbps download speed according to speedtest.net).

I've quickly verified that download speeds drop dramatically with CloudFront. So, in my opinion, the options to solve this, from best to worst ("best" meaning all devs benefit without much work), are:

  1. Set up a CDN to replace a Seattle-based endpoint that's slow for everyone outside the U.S.
  2. Mirror things to 1 or more buckets outside the U.S.
  3. Let each developer outside the U.S. run into this problem, set up their own CloudFront distribution and override BUILDPACK_VENDOR_URL

Did I miss anything obvious? What do you think of this?

schneems commented 9 years ago

Thanks for the issue, i'll take a look.

schneems commented 9 years ago

Right now all builds, even those for the EU region are happening in the US East.

The https://s3-external-1.amazonaws.com should correspond to the same data center (US East, Virginia) as the builds are happening in, so the download should be nearly instantaneous. We're measuring this download time across the platform and the 95th percentile for a download of bundler (stored in the same region) is 0.5546050 seconds with the average being 0.1 seconds.

It doesn't look like this is an issue on our end.

If you're using this buildpack outside of Heroku you can manually up the time out limit for binary downloads: https://github.com/heroku/heroku-buildpack-ruby/blob/master/lib/language_pack/fetcher.rb#L42