Vagrant box downloads extremely slow

[spkane]

When relying on vagrant to download a box I frequently see connection speeds like this:

default: Downloading: http://boxes.example.com/vagrant/boxes/c6/packer_c6_2.5.2_virtualbox.box
default: Progress: 20% (Rate: 179k/s, Estimated time remaining: 0:41:37)

(Rate: 179k/s)

Yet when I use wget to the same URL:

wget http://boxes.example.com/vagrant/boxes/c6/packer_c6_2.5.2_virtualbox.box
--2015-02-10 09:52:12--  http://boxes.example.com/vagrant/boxes/c6/packer_c6_2.5.2_virtualbox.box
Resolving boxes.example.com... 10.1.0.17
Connecting to boxes.example.com|10.1.0.17|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 830674320 (792M) [text/plain]
Saving to: 'packer_c6_2.5.2_virtualbox.box'

packer_c6_2.5.2_virtualbox.bo   0%[                                                         ]   7.12M   696KB/s   eta 19m 50s

(Rate: 696KB/s) or often higher.

This particular example was pulled when on Wifi and connected to an IPSEC VPN.

[sethvargo]

Hi @spkane

Some boxes are hosted on Atlas and sometimes Atlas is just acting as a proxy to a user-hosted box. If you give more information on the specific box(es) you're downloading, we can do some research.

[spkane]

@sethvargo This box is actually a box I built using packer and it is hosted on a remote server. I'm trying to understand why the download in significantly slower using vagrant then using wget to the exact same URL.

[sethvargo]

@spkane sorry - I misread your original issue.

I would suspect (and maybe @mitchellh could elaborate more) a few things:

1. Ruby is slow and somehow throttling the subprocess
2. Wget is faster than curl (which is what Vagrant is using)
3. Vagrant is also allocating time to unpack the box
4. Wget is allowing for some type of compressed download

It would be helpful if you could benchmark this with curl for reference.

[mitchellh]

I really can't explain this. Vagrant doesn't do anything during the subprocess Ruby-wise: it subprocesses to curl. It doesn't even do the download in Ruby. Perhaps wget is using multiple connections to download multiple parts? I really don't know, but unless we get more information I have to assume that Vagrant is fine here.

Is curl just as slow? Vagrant is just subprocessing to curl until it completes.

[keeprock]

I'm experience the same slow experience. Anyone can try aria - http://aria2.sourceforge.net/ and http://stackoverflow.com/questions/3430810/wget-download-with-multiple-simultaneous-connections

It's seems a little bit faster, but, man, you can set this up using default vagrant download mechanism and take a walk or make yourself a sandwich. Get way from screen for a little bit.

[karlkfi]

Having the same problem here:

1. Upload a box manually to atlas
2. Create a new Vagrantfile with just vm_cfg.vm.box_url = <user>/box-name
3. vagrant up - box downloads slowly
4. wget box url from atlas (see vagrant up output) - box downloads lightening fast

[clakeb]

I wish there was just a +1 for this. Me too. Same connection for all 3 attempts. VPN turned off.

• vagrant up took 25+ minutes.
• wget took 3 minutes.
• curl took 4 minutes.

[milhaus]

Ubuntu vivid64 is downloading at ~56kbps. I'm on a 100mbit symmetric connection. edit: it timed out before it could finish. edit2: I can confirm that https://atlas.hashicorp.com/ubuntu/boxes/vivid64/versions/20160128.0.0/providers/virtualbox.box downloads dramatically faster over wget than via "vagrant up".

[ghost]

I'm trying to download the scotch/box and current download speeds using vagrant are less than 10kbps.

default: Progress: 0% (Rate: 2603/s, Estimated time remaining: 33:17:38)

However just as bad using wget.

[tehmaspc]

ditto; some popular boxes are very slow to download - i'm updating ubuntu/trusty64 as we speak and it's dropping below 1Kb/s. Been seeing this for a couple wks now.

[JasonTheAdams]

+1 -- exact same as last comment

[winni2k]

Same here:

$ vagrant box add lazygray/heroku-cedar-14
==> box: Loading metadata for box 'lazygray/heroku-cedar-14'
    box: URL: https://atlas.hashicorp.com/lazygray/heroku-cedar-14
==> box: Adding box 'lazygray/heroku-cedar-14' (v1.0.6) for provider: virtualbox
    box: Downloading: https://atlas.hashicorp.com/lazygray/boxes/heroku-cedar-14/versions/1.0.6/providers/virtualbox.box
==> box: Box download is resuming from prior download progress
    box: Progress: 3% (Rate: 281k/s, ...

[k1ng440]

same here

vagrant box update
==> default: Checking for updates to 'laravel/homestead'
    default: Latest installed version: 0.4.1
    default: Version constraints: >= 0
    default: Provider: vmware_desktop
==> default: Updating 'laravel/homestead' with provider 'vmware_desktop' from version
==> default: '0.4.1' to '0.4.2'...
==> default: Loading metadata for box 'https://atlas.hashicorp.com/laravel/homestead'
==> default: Adding box 'laravel/homestead' (v0.4.2) for provider: vmware_desktop
    default: Downloading: https://atlas.hashicorp.com/laravel/boxes/homestead/versions/0.4.2/providers/vmware_desktop.box
    default: Progress: 0% (Rate: 42210/s, Estimated time remaining: 6:10:54))

[richard-scott]

Is there any way to use something like axel to stream downloads in quicker?

[winni2k]

I guess there's nothing preventing people from sharing boxes via torrent. For example, below is a magnet link for the heroku-cedar-14 box:

magnet:?xt=urn:btih:5bb1480d5316f229bb71be55b56b06278de41a67&dn=heroku-cedar-14.box&tr=http%3A%2F%2F9.rarbg.com%3A2710%2Fannounce&tr=http%3A%2F%2Fannounce.torrentsmd.com%3A6969%2Fannounce&tr=http%3A%2F%2Fbt.careland.com.cn%3A6969%2Fannounce&tr=http%3A%2F%2Fexplodie.org%3A6969%2Fannounce&tr=http%3A%2F%2Fmgtracker.org%3A2710%2Fannounce&tr=http%3A%2F%2Ftracker.tfile.me%2Fannounce&tr=http%3A%2F%2Ftracker.torrenty.org%3A6969%2Fannounce&tr=http%3A%2F%2Ftracker.trackerfix.com%2Fannounce&tr=http%3A%2F%2Fwww.mvgroup.org%3A2710%2Fannounce&tr=udp%3A%2F%2F9.rarbg.com%3A2710%2Fannounce&tr=udp%3A%2F%2F9.rarbg.me%3A2710%2Fannounce&tr=udp%3A%2F%2F9.rarbg.to%3A2710%2Fannounce&tr=udp%3A%2F%2Fcoppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fglotorrents.pw%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.glotorrents.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker4.piratux.com%3A6969%2Fannounce

Anyone know a good website where one can search for torrents of vagrant boxes?

[tehmaspc]

@wkretzsch - I personally don't know at the moment about any torrent sites - but for me I wouldn't want to trust torrent links as the source for my infrastructure testing. It's a possible option but security is also important. For me official vagrant boxes from folks like puppetlabs hosted on Atlas are so slow to download at times that I wish this issue could be resolved. For internal vagrant boxes that I build for my company we have the option to host on S3 or Artifactory or private Atlas org.

@mitchellh - yes - curl is just as slow (for me). I don't think it is a Vagrant issue - but a backed server hosting issue. Granted - not a Vagrant issue per se.

[faddat]

Yes, this is because curl can only use one of my 3 connections at the same time. No, that's not the connection's rated speed. The rated speed is 45mbps. Yes, bittorrent does perform better. Just sayin-- your rationale for not supporting bittorrent is kinda thin here.

[winni2k]

@tehmaspc surely there must be a way for a website to publish the hash of their box along with a torrent link?

[darkn3rd]

I wish in general, there was a way to have incremental images, like docker images, with vagrant boxes. For the provisioners, which bootstrap (cfengine, chef, salt, puppet, docker, etc) by downloading their platform, I wish there was a way to download a packaged up installer, so that other fresh images that use that provisioner, e.g. ubuntu + docker, would not need to download the goods again. Box updates and provisioner downloads were already painful, but recently, have been beyond notoriously slow.

[PorterBytes]

Just went to update my box for the first time (trusty64 - noticed the warning on my vagrant up command output), and it's going to take my 1.5 hours on a 150MBps connection - pathetic. It's 2016 - I don't know the specifics of what's going on here, but surely we can fix this, like, by the end of next week? The tech that goes into modern technologies like vagrant is amazing, something this basic should be overcome in mere hours.

[faddat]

Amen, Matt, Amen. This is about UX.

There should be a recognition that line speed != line speed and practical steps can be taken to overcome the daunting issue of line speed != line speed.

Jacob Gadikian E-mail: faddat@gmail.com SKYPE: faddat Phone/SMS: +84 167 789 6421

On Sun, Apr 10, 2016 at 6:32 AM, Matt Porter notifications@github.com wrote:

Just went to update my box for the first time (noticed the warning on my vagrant up command output), and it's going to take my 1.5 hours on a 150MBps connection - pathetic. It's 2016 - I don't know the specifics of what's going on here, but surely we can fix this, like, by the end of next week? The tech that goes into modern technologies like vagrant is amazing, something this basic should be overcome in mere hours.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/mitchellh/vagrant/issues/5319#issuecomment-207881297

[briancline]

I just tried asking Vagrant to download ubuntu/trusty64, and was getting speeds of <= 5 KiB/sec. I killed it and tried again using the exact same command, and got 29 MiB/sec.

I think @mitchellh is correct in that this doesn't really seem like a Vagrant issue. If anything, it seems more like an Atlas issue (so possibly the ELB and/or whatever's sitting behind it). I highly doubt it has anything to do with the routes or hops between end-users and the ELB VIPs -- you wouldn't typically see such a polarizing set of speeds in that case, especially considering both VIPs terminate in us-east-1.

If for no other reason, it'd be highly desirable to see these made available through a CDN rather than a centrally-located ELB. Then again, I'm just one guy (who isn't paying for this service), so take that for what it's worth. Pretty thankful it's there either way.

[karlkfi]

It's not just an Atlas issue. I have boxes and metadata.json on S3, with a Fastly CDN in front and regularly have the exact same issue: sometimes vagrant downloads at 100kbps and sometimes it downloads at > 5mbps. You can cancel a slow download and half the time a retry gets you the faster speeds.

[milhaus]

I contacted support about this around the same time I chimed in here initially. Their response is that Vagrant uses curl to download things so they don't see this as a Vagrant problem. IMO that's an unprofessional cop-out because they chose to use curl, know that there are problems and aren't considering swapping out with an alternative to eliminate the problem for their users.

[Haroenv]

I can confirm that this is still an issue. All my peers also report times of >1h, while the connection here for other connections is around 200MB/s.

vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'ubuntu/trusty32' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Loading metadata for box 'ubuntu/trusty32'
    default: URL: https://atlas.hashicorp.com/ubuntu/trusty32
==> default: Adding box 'ubuntu/trusty32' (v20160406.0.0) for provider: virtualbox
    default: Downloading: https://atlas.hashicorp.com/ubuntu/boxes/trusty32/versions/20160406.0.0/providers/virtualbox.box
    default: Progress: 11% (Rate: 43801/s, Estimated time remaining: 1:36:50)

[faddat]

While I am unsure of the origin of the problem, I really do wish that Hashicorp would get back to its unrelenting focus on user experience with this one. Muli-hour downloads (that should take 1-10 minutes)==bad ux.

[dacodekid]

Currently downloading an image for the 5th time (@13Xk/s, even with wget). Keep disconnecting me while around 50-90%. But it ALWAYS downloads at full speed either early morning / late night EST. Assuming it is a traffic issue, but regardless very bad UX.

    box: Progress: 47% (Rate: 106k/s, Estimated time remaining: 0:14:50)

[brazitech]

I have been trying for 2 day's now and still can not get it to download... its a shame.. it is really not impressing new comers to laravel .. i can only get 34ks speed.........

[richard-scott]

Speeds ok from the UK:

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'bento/centos-7.2' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Loading metadata for box 'bento/centos-7.2'
    default: URL: https://atlas.hashicorp.com/bento/centos-7.2
==> default: Adding box 'bento/centos-7.2' (v2.2.6) for provider: virtualbox
    default: Downloading: https://atlas.hashicorp.com/bento/boxes/centos-7.2/versions/2.2.6/providers/virtualbox.box
    default: Progress: 11% (Rate: 7728k/s, Estimated time remaining: 0:01:19)

What is your location?

[richard-scott]

Also, https://atlas.hashicorp.com/ URL's are delivered from Amazon Web Services (atlas-frontend-atlas-230110478.us-east-1.elb.amazonaws.com) so I doubt they are tight for bandwidth ;-)

Are the slow downloads being made from locations a long distance away from the AWS us-east-1 DC, perhaps thats the root cause of the issue?

Maybe the AWS CDN could be used to cache files around the world?

[PorterBytes]

I'm located in Vermont, which is pretty us-east-1 last I checked :dart:

[brazitech]

I am in Brazil.... got it to download.... 10m connection here took 4.6 hours!!!!! my wife just gave birth to our 8th little girl.. It only took her 40 minutes !!!!! lol There is a big problem with there download!!!!!

[thokari]

Same here, I have a 50Mbps connection...

default: Progress: 44% (Rate: 102k/s, Estimated time remaining: 0:15:01))

[macnibblet]

Sign me upp here, 100mb symmetric connection (Fiber) sloooow as shit, doing 150kb/s

[brazitech]

after looking at the years of complaints of slow download with no effort of resolving the issue,,, i think its time to start emailing Laravel to stop endorsing homestead until the issue is resolved..... maybe that will get their attention!!!! this is a real problem... 15 retries and then 4.6 hours to download a file is irresponsible on their part........

[richard-scott]

I bet it gets closed, but if you don't ask you don't get:

https://github.com/mitchellh/vagrant/issues/7307

[bradisbell]

Just snagged a box at 85mbit. You all fix something recently? Much better than it used to be.

[voycey]

This is painful to do anything on any more - On 100mbps synchronous connection and getting 168kb, either overloaded servers or throttling

[mccorkle]

In looking to debug curl being slow -- I found a stackoverflow post that suggests that --trace-ascii /dev/null makes your curl go at the speed you'd expect. For me, I'm trying to download CentosOS 7 and here are my results:

NO --trace-ascii option:

$ curl http://cloud.centos.org/centos/nt/x86_64/images/CentOS-7-x86_64-Vagrant-1606_01.VirtualBox.box -o CentOS-7-x86_64-Vagrant-1606_01.VirtualBox.box
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  483M    0  489k    0     0  77724      0  1:48:44  0:00:06  1:48:38 69721

With trace-ascii the first time:

$ curl http://cloud.centos.org/centos/7/vagrant/x86_64/images/CentOS-7-x86_64-Vagrant-1606_01.VirtualBox.box -o CentOS-7-x86_64-Vagrant-1606_01.VirtualBox.box --trace-ascii /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  5  483M    5 24.5M    0     0  4259k      0  0:01:56  0:00:05  0:01:51 4599k

Does anyone else see the same behavior?

[codehaiku]

The download is extremely slow on my end too. I'm trying vagrant for the very first time. Might ditch this software and go back to my native apache2 instead.

[daryn-k]

Help. I have same problem. I can't wait 3 hours! Very slow! Stupid!

[brazitech]

i gave up a year ago..download to slow.. problems after down load.. have to download for 3 hours again.... Vagrant will not fix the problem that has been there for several years now.. you would think that after 3 or 4 years of this problem they would address the issue.....

[daryn-k]

8 hours to download! I hate you all!

[winni2k]

Lol, I hate you too @daryn-k :)

[ameyaagashe]

Guys why is this issue closed? This is still an outstanding issue and needs to be addressed ASAP. I am experiencing the same issue.

[grassjedi]

Wow! Downloading boxes is painful please fix this. PLEASE?

[JasonTheAdams]

I think it really has to do with time of day, traffic, alignment of the planets, etc. I haven't had slow speeds in a while. It seems very hit or miss. In fact, if it is slow you and start and stop it with the chance of getting a better connection. I'm not sure this is really the fault of the vagrant framework as much as it is the nature of large bottlenecked downloads.

[ryanaslett]

@mitchellh These errant speed symptoms from hashicorp's servers could be indicative of bumping up against AWS's IOPS credits for GP2 filesystems. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html#IOcredit

We had some testing infrastructure on drupal.org that would run fine for a long time, then suddenly drop to a crawl because we had "spent" all of our IO credits. It could be possible that hashicorps' servers are bumping up against the same limit.

sar -b could give some insight as to whether or not this explains the random performance drops.

[richard-scott]

It could be an idea for Hashicorp to move the images for download into S3 and use that for downloads... that would save on running instances specifically for downloads.

[babadofar]

Downloading boxes used to be quick, now it's so slow it makes vagrant a no-go for quick and simple developer environments.