Gems are not cached #62

Closed tuminoid closed 10 years ago

I'm using cachier 0.5.0 with VMWare. I have set bucket auto-detect, NFS and single machine scope. Apt caching works perfectly , but the gem cache is always empty. I have set cachier to auto-detect.

For example, if I install GitLab with my installer, apt cache is at 169M, and gem cache is 0M, even GitLab installs a huge bunch of them from rubygems.org.

Correct me if I'm wrong but you are using bundle install --deployment which installs gems to vendor/bundle. As stated on the README, the plugin hooks into the gem env gemdir path :)

TBH, we really can't cache gems if bundle install --deployment is used apart from having support for caching specific directories which will eventually be handled by GH-4.

I'm going to keep this open and tagged as a documentation improvement since I believe you are not the first to ask that. Thanks for reporting

Moi, Tuomo =)

@fgrehm wrote:

we really can't cache gems if bundle install --deployment is used

But doesn't it still use the normal cache directory?

Anyway I think the main issue here is that the same provisioning run installs both ruby and the gems. And vagrant-cachier won't configure the gem cache as gem command is not found on the VM (before the provisioning). @tuminoid, try installing ruby in a separate provisioner run before the main one.

@tmatilai nope =/ I just double checked that and at least based on my tests it doesn't use the cache: https://gist.github.com/fgrehm/a87a8f9aa0753b524fa3. But what you said also makes sense because even if the --deployment is omitted, if ruby is installed on the same provisioning step as the bundle install, cachier won't be able to find out the directory to hook into.

@fgrehm I stand corrected. Sorry for being too busy (read: lazy) to test myself. =) Maybe that behavior even makes sense, as it allows you to install dependencies without having write access to any other directories. Strange still...

Another tricky part with gem bucket is when using custom ruby packages/installations and custom GEM_HOME etc. As vagrant-cachier only finds the gem in PATH, these are not cached either. Maybe #4 is the best road here too.

@tmatilai no worries, it happens to me all the time too :P

@tuminoid just to sum up, in order to benefit from this plugin you'll need to do 2 things while we don't get to GH-4:

Install ruby on a separate provisioner
Remove the --deployment flag from bundle install on your scripts

If you can't / don't want to go that direction, you might want to check out @tmatilai's vagrant-proxyconf so that caching is transparent to your provisioning scripts. I'm not 100% sure it will work for rubygems but might be worth giving it a shot if you can't change your scripts.

Hope that helps =)

Setting environment variables via vagrant-proxyconf does indeed work with gem. And if I read the script correctly, it uses HTTP (without SSL) to access rubygems.org, so you can use a caching http proxy (e.g. polipo-box).

For security reasons it is nowadays recommended to use HTTPS for rubygems.org, and then the proxy should do some nasty man-in-the-middle hacks to be able to cache the traffic. I recently found some interesting docs for squid; have to play with it. Okay, getting a bit OT here. :)

@fgrehm Thanks for excellent explanation of the situation. I will tinker with the options given to see if there is easy value available right now. If not, I'm happy to wait for GH-4 to resolve in future.

@tmatilai I have chosen to use HTTP as default (with a setting to change it back) simple because their SSL certs are consistently yet randomly failing on Ubuntu and that is not OK in non-interactive provisioners. There is ways to alleviate those in production installs, but I'm unwilling to put in code to workaround them in all provisioner's with accessing their site.

fgrehm / vagrant-cachier

Gems are not cached #62