Copy gemspec to /usr/src/app

[msabramo]

before bundle install.

This is:

• simpler

• solves problem that some Gemfiles reference a gemspec (See Yehuda Katz post) and this wasn't including the gemspec before running bundle install, resulting in:

  Trigger 2, RUN bundle install --system
  Step 0 : RUN bundle install --system
  ---> Running in 2cb8961c7e75
  Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
  installing your bundle as root will break this application for all non-root
  users on this machine.
  There are no gemspecs at /usr/src/app.
  2015/02/02 20:53:06 The command [/bin/sh -c bundle install --system] returned a non-zero code: 15

Sample Gemfile:

source 'http://rubygems.org'

ruby '1.9.3', engine: 'jruby', engine_version: '1.7.19'

gemspec

[cpuguy83]

This pattern is taken from the official ruby-onbuild image. The reason it is this way is to make the most effective use of the build cache.

In your version, in cases where something in "." has changed (anything at all), all gems will have to be re-installed. This is undesirable in almost all scenarios.

It does make since to add the gempsec in before running bundle install. If you want to change your PR to that I'd be happy to merge.

cc @tianon @yosifkit

[msabramo]

Ah, right. Forgot about the build cache. I'll update shortly.

Is there a similar reason for putting ADDs on separate lines -- e.g.:

ONBUILD ADD Gemfile /usr/src/app/
ONBUILD ADD Gemfile.lock /usr/src/app/

instead of say:

ONBUILD ADD Gemfile Gemfile.lock *.gemspec /usr/src/app/

?

[cpuguy83]

Because it wasn't possible before. Let's keep it separate for now.

[msabramo]

Ok, I'll update this in a bit.

[cpuguy83]

Thanks!

[msabramo]

Updated. 7faa356 just adds the one line with the gemspecs and 2fdb681 adds a comment that explains the ordering of steps to optimize use of the build cache.

$ docker build -t msabramo/jruby:1.7-onbuild .
Sending build context to Docker daemon 3.072 kB
Sending build context to Docker daemon
Step 0 : FROM jruby:1.7-jdk
 ---> 6a31a0c9b2b2
Step 1 : RUN mkdir -p /usr/src/app
 ---> Using cache
 ---> 8a45d7ddcc4f
Step 2 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 58e75d850ff4
Step 3 : ONBUILD add Gemfile /usr/src/app/
 ---> Using cache
 ---> debd9c9655c3
Step 4 : ONBUILD add Gemfile.lock /usr/src/app/
 ---> Using cache
 ---> e7de343150bf
Step 5 : ONBUILD add *.gemspec .gemspec /usr/src/app/
 ---> Running in 7d5e876014f7
 ---> bde24e0f1290
Removing intermediate container 7d5e876014f7
Step 6 : ONBUILD run bundle install --system
 ---> Running in 23f1aece14eb
 ---> 0890c4dba0ec
Removing intermediate container 23f1aece14eb
Step 7 : ONBUILD add . /usr/src/app
 ---> Running in 0fb8ad08f1b3
 ---> 6648cafb06c6
Removing intermediate container 0fb8ad08f1b3
Successfully built 6648cafb06c6

[msabramo]

Gah.

$ docker build -t msabramo/docverter .
Sending build context to Docker daemon 4.708 MB
Sending build context to Docker daemon
Step 0 : FROM msabramo/jruby:1.7-onbuild
# Executing 5 build triggers
Trigger 0, ADD Gemfile /usr/src/app/
Step 0 : ADD Gemfile /usr/src/app/
Trigger 1, ADD Gemfile.lock /usr/src/app/
Step 0 : ADD Gemfile.lock /usr/src/app/
Trigger 2, ADD *.gemspec .gemspec /usr/src/app/
Step 0 : ADD *.gemspec .gemspec /usr/src/app/
2015/02/03 07:44:45 .gemspec: no such file or directory

How do I add files that may or may not be there?

[msabramo]

It looks like changing .gemspec to .gemspec* solves that problem.

[msabramo]

Updated. 63d5d4077645a54947628d9a442f781169d416b4 just adds the one line with the gemspecs and de90ceb7ee22d4825deeeed7ba2d3f636b010ed6 adds a comment that explains the ordering of steps to optimize use of the build cache.

$ docker build -t msabramo/jruby:1.7-onbuild .
Sending build context to Docker daemon 3.072 kB
Sending build context to Docker daemon
Step 0 : FROM jruby:1.7-jdk
 ---> 6a31a0c9b2b2
Step 1 : RUN mkdir -p /usr/src/app
 ---> Using cache
 ---> 8a45d7ddcc4f
Step 2 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 58e75d850ff4
Step 3 : ONBUILD add Gemfile /usr/src/app/
 ---> Using cache
 ---> debd9c9655c3
Step 4 : ONBUILD add Gemfile.lock /usr/src/app/
 ---> Using cache
 ---> e7de343150bf
Step 5 : ONBUILD add *.gemspec .gemspec* /usr/src/app/
 ---> Running in 8d86a6f3eb11
 ---> e4e68c069b58
Removing intermediate container 8d86a6f3eb11
Step 6 : ONBUILD run bundle install --system
 ---> Running in 8473cdd4f757
 ---> 6cf25bb788de
Removing intermediate container 8473cdd4f757
Step 7 : ONBUILD add . /usr/src/app
 ---> Running in 94d183502d94
 ---> fd20142d559f
Removing intermediate container 94d183502d94
Successfully built fd20142d559f

[msabramo]

Hmmm, running into an interesting problem trying to build Docverter as a Docker image.

The docverter.gemspec has this line:

require 'docverter-server/version'

so for this to work, we need lib/docverter-server/version.rb at the time of running bundle install but it doesn't get added until later. How would one handle this?

I guess this worked with my first version that wasn't build-cache friendly because that added everything before bundle install. But how do we solve this in a build-cache-friendly way?

[cpuguy83]

Assuming that your gemspec is referring to the version in version.rb?

[msabramo]

Yep.

[msabramo]

And perhaps a valid answer is "Don't reference files from your gemspec". I am not the creator/maintainer of Docverter though so I can't make that happen.

[cpuguy83]

This is probably going to be fairly normal to do, I don't think we can add this.

If the onbuild image isn't quite working for you, then best to just use the normal image.

[tianon]

https://github.com/docker-library/ruby/issues/22

[msabramo]

Ok. Going to do a quick local hack to Docverter just so that I can test out the new and improved jruby image:

--- a/docverter.gemspec
+++ b/docverter.gemspec
@@ -1,10 +1,9 @@
 lib = File.expand_path("../lib", __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
-require 'docverter-server/version'

 Gem::Specification.new do |gem|
   gem.name          = "docverter-server"
-  gem.version       = DocverterServer::VERSION
+  gem.version       = "1.0.3"
   gem.authors       = ["Pete Keen"]
   gem.email         = ["pete@docverter.com"]
   gem.description   = %{Document conversion service with a REST API}

With that:

$ docker build -t msabramo/docverter .
Sending build context to Docker daemon  6.77 MB
Sending build context to Docker daemon
Step 0 : FROM jruby:1.7-onbuild
# Executing 5 build triggers
Trigger 0, ADD Gemfile /usr/src/app/
Step 0 : ADD Gemfile /usr/src/app/
 ---> Using cache
Trigger 1, ADD Gemfile.lock /usr/src/app/
Step 0 : ADD Gemfile.lock /usr/src/app/
 ---> Using cache
Trigger 2, ADD *.gemspec .gemspec* /usr/src/app/
Step 0 : ADD *.gemspec .gemspec* /usr/src/app/
Trigger 3, RUN bundle install --system
Step 0 : RUN bundle install --system
 ---> Running in dd5197959010
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from http://rubygems.org/.........
Resolving dependencies...
...
Successfully built dc73a0585f46

so I'm happy with this.

[yosifkit]

I would be -1 here. If a user has no gemspec files (only Gemfile and lock) then it will fail to build for them.

[cpuguy83]

@yosifkit It seems to work if you use the COPY foo* method instead of directly specifying the file. However as was noticed above, most people refer to a VERSION const in their gemspecs, and there is no way for us to programmatically add that here.

[msabramo]

On a phone so can't check now, but I think it will work if the ADD is done in one command because as long as there is at least one match I think it's ok.

ONBUILD ADD Gemfile Gemfile.lock *.gemspec .gemspec* /usr/src/app/

Should I change it to do that?

[cpuguy83]

Closing as this will have undesirable consequences for users missing a gemspec, or using a version of docker that doesn't support wildcards.