vagrant-cachier not caching

[amitoj]

Hi,

I installed vagrant-cachier and configured my Vagrantfile as,

... ... Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.box = "precise64"

if Vagrant.has_plugin?("vagrant-cachier") config.cache.scope = :box end

config.vm.box_url = "http://files.vagrantup.com/precise64.box"

... ...

I'm downloading and configuring a Zookeeper server and a Kafka server. Both VMs need to download Java and Kafka binaries.

I expected vagrant to pick cached binaries the second time but it is downloading them all over again.

==> zookeeper: Location: http://download.oracle.com/otn-pub/java/jdk/8u20-b26/jdk-8u20-linux-x64.tar.gz?AuthParam=1409755263_9100faf0885fc01894af83fe3915e3f9 [following] ==> zookeeper: --2014-09-03 14:39:03-- http://download.oracle.com/otn-pub/java/jdk/8u20-b26/jdk-8u20-linux-x64.tar.gz?AuthParam=1409755263_9100faf0885fc01894af83fe3915e3f9 ==> zookeeper: Connecting to download.oracle.com (download.oracle.com)|109.144.113.145|:80... ==> zookeeper: connected. ==> zookeeper: HTTP request sent, awaiting response... ==> zookeeper: 200 OK ==> zookeeper: Length: 160872342 (153M) [application/x-gzip] ==> zookeeper: Saving to: jdk-8u20-linux-x64.tar.gz' ==> zookeeper: ==> zookeeper: 0K . ==> zookeeper: . ==> zookeeper: . ==> zookeeper: . ... ... ... ==> zookeeper: . ==> zookeeper: . 99% 464K 1s ==> zookeeper: ==> zookeeper: 156672K . ==> zookeeper: . ==> zookeeper: . ==> zookeeper: . ==> zookeeper: . ==> zookeeper: . ==> zookeeper: 100% 407K=5m49s ==> zookeeper: ==> zookeeper: 2014-09-03 14:44:52 (450 KB/s) -jdk-8u20-linux-x64.tar.gz' saved [160872342/160872342] ==> zookeeper: Download done.

==> brokerOne: Location: http://download.oracle.com/otn-pub/java/jdk/8u20-b26/jdk-8u20-linux-x64.tar.gz?AuthParam=1409755850_5d0be48911525e66191510ed48130abf [following] ==> brokerOne: --2014-09-03 14:48:51-- http://download.oracle.com/otn-pub/java/jdk/8u20-b26/jdk-8u20-linux-x64.tar.gz?AuthParam=1409755850_5d0be48911525e66191510ed48130abf ==> brokerOne: Connecting to download.oracle.com (download.oracle.com)|109.144.113.145|:80... ==> brokerOne: connected. ==> brokerOne: HTTP request sent, awaiting response... ==> brokerOne: 200 OK ==> brokerOne: Length: 160872342 (153M) [application/x-gzip] ==> brokerOne: Saving to: jdk-8u20-linux-x64.tar.gz' ==> brokerOne: ==> brokerOne: 0K . ... ... ... ==> brokerOne: . 99% 475K 1s ==> brokerOne: ==> brokerOne: 156672K . ==> brokerOne: . ==> brokerOne: . ==> brokerOne: . ==> brokerOne: . ==> brokerOne: . ==> brokerOne: 100% 476K=5m33s ==> brokerOne: ==> brokerOne: 2014-09-03 14:54:24 (472 KB/s) -jdk-8u20-linux-x64.tar.gz' saved [160872342/160872342] ==> brokerOne: Download done.

The contents of /Users/amitojsh/.vagrant.d/cache/precise64/apt directory are,

macbook:apt amitojsh$ ll total 67984 -rw-r--r-- 1 amitojsh staff 137742 14 Apr 19:08 curl_7.22.0-3ubuntu4.8_amd64.deb -rw-r--r-- 1 amitojsh staff 6953830 16 Apr 2012 g++-4.6_4.6.3-1ubuntu5_amd64.deb -rw-r--r-- 1 amitojsh staff 1442 13 Mar 2012 g++_4%3a4.6.3-1ubuntu5_amd64.deb -rw-r--r-- 1 amitojsh staff 630012 11 Apr 2012 git-man_1%3a1.7.9.5-1_all.deb -rw-r--r-- 1 amitojsh staff 6086854 11 Apr 2012 git_1%3a1.7.9.5-1_amd64.deb -rw-r--r-- 1 amitojsh staff 9108 30 Apr 2011 gsfonts-x11_0.22_all.deb -rw-r--r-- 1 amitojsh staff 3373970 21 Sep 2010 gsfonts_1%3a8.11+urwcyr1.0.7~pre44-4.2ubuntu1_all.deb -rw-r--r-- 1 amitojsh staff 61704 2 Dec 2011 java-common_0.43ubuntu2_all.deb -rw-r--r-- 1 amitojsh staff 236614 14 Apr 19:08 libcurl3_7.22.0-3ubuntu4.8_amd64.deb -rw-r--r-- 1 amitojsh staff 23762 3 Dec 2007 liberror-perl_0.17-1_all.deb -rw-r--r-- 1 amitojsh staff 15432 20 Nov 2010 libfontenc1_1%3a1.1.0-1_amd64.deb -rw-r--r-- 1 amitojsh staff 17158 18 Oct 2011 libgpm2_1.20.4-4_amd64.deb -rw-r--r-- 1 amitojsh staff 1188308 3 Mar 2014 libpython2.7_2.7.3-0ubuntu3.5_amd64.deb -rw-r--r-- 1 amitojsh staff 1659636 16 Apr 2012 libstdc++6-4.6-dev_4.6.3-1ubuntu5_amd64.deb -rw-r--r-- 1 amitojsh staff 134062 14 May 15:13 libxfont1_1%3a1.4.4-1ubuntu0.2_amd64.deb -rw-r----- 1 amitojsh staff 0 14 Sep 2012 lock -rw-r--r-- 1 amitojsh staff 119000 12 Nov 2012 make_3.81-8.1ubuntu1.1_amd64.deb -rw-r--r-- 1 amitojsh staff 21908 20 Aug 15:59 oracle-java8-installer_8u20+8u6arm-1~webupd8~0_all.deb -rw-r--r-- 1 amitojsh staff 4366 20 Aug 15:59 oracle-java8-set-default_8u20+8u6arm-1~webupd8~0_all.deb drwxr-xr-x 2 amitojsh staff 68 3 Sep 15:47 partial -rw-r--r-- 1 amitojsh staff 80174 31 Jan 2012 patch_2.6.1-3_amd64.deb -rw-r--r-- 1 amitojsh staff 49154 31 Dec 2011 python-pycurl_7.19.0-4ubuntu3_amd64.deb -rw-r--r-- 1 amitojsh staff 23454 19 Dec 2013 python-software-properties_0.82.7.7_all.deb -rw-r--r-- 1 amitojsh staff 1742642 3 Mar 2014 python2.7-minimal_2.7.3-0ubuntu3.5_amd64.deb -rw-r--r-- 1 amitojsh staff 2674922 3 Mar 2014 python2.7_2.7.3-0ubuntu3.5_amd64.deb -rw-r--r-- 1 amitojsh staff 611204 6 Jun 2011 screen_4.0.3-14ubuntu8_amd64.deb -rw-r--r-- 1 amitojsh staff 8450 19 Dec 2013 software-properties-common_0.82.7.7_all.deb -rw-r--r-- 1 amitojsh staff 24654 15 Nov 2012 unattended-upgrades_0.76ubuntu1_all.deb -rw-r--r-- 1 amitojsh staff 86498 4 May 2012 vim-common_2%3a7.3.429-2ubuntu2.1_amd64.deb -rw-r--r-- 1 amitojsh staff 6276884 4 May 2012 vim-runtime_2%3a7.3.429-2ubuntu2.1_all.deb -rw-r--r-- 1 amitojsh staff 410888 4 May 2012 vim-tiny_2%3a7.3.429-2ubuntu2.1_amd64.deb -rw-r--r-- 1 amitojsh staff 1048020 4 May 2012 vim_2%3a7.3.429-2ubuntu2.1_amd64.deb -rw-r--r-- 1 amitojsh staff 279358 23 Jun 19:28 wget_1.13.4-2ubuntu1.1_amd64.deb -rw-r--r-- 1 amitojsh staff 51986 3 Jan 2013 x11-common_1%3a7.6+12ubuntu2_all.deb -rw-r--r-- 1 amitojsh staff 583178 19 Mar 2012 xfonts-encodings_1%3a1.0.4-1ubuntu1_all.deb -rw-r--r-- 1 amitojsh staff 96406 30 Jun 2011 xfonts-utils_1%3a7.6+1_amd64.deb

Any ideas?

Thanks,

Amitoj.

[fgrehm]

Apparently your VM is not installing Java using debs as it is fetching it from http://download.oracle.com/otn-pub/java/jdk/8u20-b26/jdk-8u20-linux-x64.tar.gz?AuthParam=1409755850_5d0be48911525e66191510ed48130abf

Would you be able to share the scripts / recipes you are using to configure your VM?

[opticyclic]

I have noticed this too. Fortunately I had originally tested out vagrant cachier in January so I knew it worked with my project at some point.

Vagrant File:

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  # Every Vagrant virtual environment requires a box to build off of.
  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"
  config.vm.hostname = "testvm"

  if Vagrant.has_plugin?("vagrant-cachier")
    config.cache.auto_detect = true
  end  

  config.vbguest.auto_update = false

  config.vm.provision :puppet do |puppet|
    puppet.module_path = "modules"
    puppet.manifests_path = "manifests"
    puppet.manifest_file  = "test_vm.pp"
  end
end

Puppet File

class system-update()
{
  exec { 'apt-update':
    command => '/usr/bin/apt-get -y update',
    timeout => 3600;
  }

  package {
    ['dos2unix', 'unzip', 'vim']:
    ensure => installed,
  }

}

It creates the caches with version 0.5.1 but not with 1.0.0.

[fgrehm]

@opticyclic I believe your problem is a different one as you haven't set the cache scope, that was a backwards incompatible change introduced on 0.6.0 with GH-17 ;-)

[amitoj]

Hi @fgrehm

My Vagrantfile is,

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  if Vagrant.has_plugin?("vagrant-cachier")
      config.cache.scope = :box
  end

  config.vm.box = "hashicorp/precise64"

  config.vm.define "test" do |test|
    test.vm.network :private_network, ip: "192.168.86.5"
    test.vm.hostname = "test"
    test.vm.provider :virtualbox do |vb|
      vb.customize ["modifyvm", :id, "--memory", "2048"]
    end
    test.vm.provision "shell", path: "test.sh"
  end

  config.vm.define "zest" do |zest|
    zest.vm.network :private_network, ip: "192.168.86.6"
    zest.vm.hostname = "zest"
    zest.vm.provider :virtualbox do |vb|
      vb.customize ["modifyvm", :id, "--memory", "2048"]
    end
    zest.vm.provision "shell", path: "test.sh"
  end

end

and, test.sh

#!/bin/sh
apt-get -y update
apt-get install -y software-properties-common python-software-properties
add-apt-repository -y ppa:webupd8team/java
apt-get -y update
/bin/echo debconf shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections
apt-get -y install oracle-java8-installer oracle-java8-set-default

It would be nice to be able to cache the Oracle's JDK as it's the most common download for all my vagrant environments.

Thanks,

Amitoj

[rafaelri]

As you mentioned precise64 box, I am using precise32 box and I have the feeling that it is not being able to properly cache its debs using vagrant cachier since its postinstall.sh script calls "apt-get -y clean" and I have the feeling it erases apt cache folder thus reflecting in NFS share used to cache content for subsequent calls.

[fgrehm]

@rafaelri yup, apt-get clean will clean the cache.

@amitoj its been a while since you opened this issue, is it still a problem for you?

[nicbet]

Any updates on this? I ran into the same problem - the .deb files are properly cached but as part of the install the oracle-java-8 deb will download a tar.gz from download.oracle.com, which does not appear to get cached, or gets deleted in some clean-up step.

[fgrehm]

Not that I know, this plugin is currently looking for maintainers: https://github.com/fgrehm/vagrant-cachier/issues/143

[fgrehm]

Hey, sorry for the silence here but this project is looking for maintainers :sweat_smile:

As per https://github.com/fgrehm/vagrant-cachier/issues/193, I've added the ignored label and will close this issue. Thanks for the interest in the project and LMK if you want to step up and take ownership of this project on that other issue :wave: