support for bundler

[rubiii]

hey guys,

first of all: this sounds like a great project and i'd love to participate!

my questions is: does rubygems-test support bundler? i feel like a growing number of gems come with a gemfile for easy setup. sometimes not all development/test dependencies can be declared in the gemspec, but the gemfile seems to be a good place for doing this.

so (i don't know if it does), but i think it's a good idea for the rubygems-test gem to embrace the gemfile and install dependencies required for development/testing.

[guillermo]

I don't like gems with bundler. Bundler have sense on final software, but not in libraries distributed as gems where the way of adding dependencies is through the gemspec as you can see in the docs:

http://docs.rubygems.org/read/chapter/20#dependencies http://docs.rubygems.org/read/chapter/20#development_dependencies

By default when you install a gem all runtime dependencies are installed, and if you want to install development dependencies, as the gem command reference says:

$ gem help install
--development                Install any additional development dependencies

So I don't found any good reason to use bundler inside a library and i can't see any good reason to add suport for bundler to add a functionality that current rubygems already have.

[rubiii]

it's not a matter of whether you like it or not. as i mentioned before, rubygems doesn't support optional dependencies. but gems with optional dependencies may require at least one of these for executing the tests. all bundler does is help installing both development and optional dependencies.

[guillermo]

Sorry. I don't know why i miss some part of your message.

I think, the actuall system may work, because what rubygems-test do is call to rake test.

Inside your rake test task you could call bundler or anything you want to prepare your gem to be tested. So i can't understand which kind of explicit support do you want.

And sorry again for my last comment.

[rubiii]

no problem guillermo, i'll try and report back on how it worked. thanks!

[rubiii]

so, i've tested and reported the test results for one my of projects: http://gem-testers.org/gems/savon

savon uses bundler to install an optional dependency (an http client) for httpi: https://github.com/rubiii/savon/blob/v0.8.5/Gemfile

and even though httpi falls back to use net/http if any of the more advanced http clients is missing, i think here are still some other gems (without fallbacks) requiring users to install one of their optional dependencies. and maybe they're also using bundler to let people easily execute their test suites.

also since i (and probably many other gem authors) use bundler to set up their test environment, the tests will fail when a user hasn't installed the bundler gem. i'm not judging ... just mentioning that this is not a real "fail".

also if they not only use the gemspec argument in their gemfile but also mention additional (probably optional) dev/test dependencies, these won't be installed, which also causes the tests to fail (the bundler setup usually only requires the specified gems but does not install them).

i think rubygems-test is a great idee/project and would love to see it succeed. but i also think it's worth thinking about these rather specific use cases.

thanks!

[rubiii]

bundler. again ;) i cloned rubygems-test because i caught a bug which i'd like to fix. now to install its dev/test dependencies (fortunately it only depends on hoe), i have to install them one by one (manually) or install the rubygems-test gem again passing --development.

i think most people using ruby also know rails. and a growing number of people using rails3 also know and use bundler. so, it's way easier for them (and me) to just call bundle and let bundler install the dev/test deps. just saying.

ps. i'm kinda blocked by JRUBY-5442 to start investigating the bug i mentioned before. in case anyone else succeeds ... it just said can't dup Symbol after i ran the following command:

jruby --1.9 -S gem test savon
...
264 examples, 0 failures
Upload these results? [Yn]  Y
can't dup Symbol

also i'd like to mention that running jruby --1.9 -S rake test within my project did not throw that error. so i think there's a problem in rubygems-test.

[guillermo]

I am not a developer a rubygems-test, but in that case, if you use bundler for development, you always can put bundler (and may the version of bundler you needed) in the gemspec development dependencies.

As i can see in the code of rubygems-test, it install any development dependency. I think you can put bundler as a development dependency and run bundler install from your rake test task.

[rubiii]

thanks. sounds like a reasonable solution. i feel like this would affect quite a few people following the same pattern ... but ... let me sleep on it.

[erikh]

@rubiii: we use hoe so you can 'hoe newb' to install your gems, or you can do it the manual way; I think hoe has a dependency on sudo right now that might not work well if you're in an rvm'd environment.

@guillermo is completely correct on the bundler development dependency. If you depend on it to run your tests, it should definitely be in there.

I'll answer the original question in the next comment.

[erikh]

Bundler support is technically there, and as of 1.0.10 and rubygems 1.5.0 fully supports the UI features it rubygems-test requires.

The way it works though, and bundler's purpose as an application, do not really jive with rubygems-test as a whole though, and therefore I would recommend just testing your gems with the comments I've made above.

If you do wish to do it this way, our API is far from rigid at this point but there's nothing stopping someone from writing a competing client (which is all rubygems-test really is) that worked more appropriately to the bundler style of development. I would be happy to work with you on any solutions you came up with to maintain compatibility and so on.

[erikh]

A client that submitted to gem-testers.org, that is.

[rubiii]

erik, i'm not complaining about hoe (even though i think it's way too easy but build and publish gems without this abstraction). it's just that i have to reinstall rubygems-test with the development option again to execute its tests. if you're cloning another gem which has like five dev dependencies and you haven't installed it using the --development argument, you would get five errors saying "ohh didn't notice? i need dependency x". in that case, having a gemfile and being able to call just one command to install these dev dependencies (which bundler pulls right out of your gemspec) is great.

also, please note that the only reason i'm "complaining" is because i care! i really like the project and i don't have a final solution ... but i also don't think i will be the last person to ask you to think about this problem.

[erikh]

I wasn't angry, and I appreciate the comments.

That said, rubygems-test already installs dev dependencies, taken right out of your gemspec. I'm not quite sure what the problem is at this point.

[myronmarston]

I'm also having issues because my gem uses bundler. Bundler is important for my dev process on my gem because I test my gems against 7 different ruby interpreters, and my gem has tons of dev dependencies (17 at the moment, and some of those have additional dependencies). Using bundler makes it easy to have a single script that I can run that uses rvm to use each supported ruby interpreter, run bundle install and run all tests.

So here's the problem I'm having. My Rakefile requires bundler and I have bundler declared as a dev dependency in my gemspec. When I run gem test mygem it goes through and asks me if I want to install each dev dependency; I answered yes to each of those (including bundler when it asked), but when it comes time to run the tests, the require bundler fails because it appears not be installed (even though it told me it was installing it...). Here's the output I'm getting:

➜  ~  gem install vcr-gem-testers
Fetching: vcr-gem-testers-1.5.1.gem (100%)
Successfully installed vcr-gem-testers-1.5.1
1 gem installed
Installing ri documentation for vcr-gem-testers-1.5.1...
Installing RDoc documentation for vcr-gem-testers-1.5.1...
➜  ~  gem test vcr-gem-testers
Install development dependency faraday (~> 0.5.3)? [Yn]  Y
Installing test dependency faraday (~> 0.5.3)
Install development dependency fakeweb (~> 1.3.0)? [Yn]  Y
Installing test dependency fakeweb (~> 1.3.0)
Install development dependency httpclient (~> 2.1.5.2)? [Yn]  Y
Installing test dependency httpclient (~> 2.1.5.2)
Install development dependency timecop (~> 0.3.5)? [Yn]  Y
Installing test dependency timecop (~> 0.3.5)
Install development dependency shoulda (~> 2.9.2)? [Yn]  Y
Installing test dependency shoulda (~> 2.9.2)
Install development dependency rspec (~> 2.4.0)? [Yn]  Y
Installing test dependency rspec (~> 2.4.0)
**************************************************

  Thank you for installing rspec-core-2.4.0

  Please be sure to look at the upgrade instructions to see what might have
  changed since the last release:

  http://github.com/rspec/rspec-core/blob/master/Upgrade.markdown

**************************************************
Install development dependency rack (= 1.1.0)? [Yn]  Y
Installing test dependency rack (= 1.1.0)
Install development dependency aruba (~> 0.2.1)? [Yn]  Y
Installing test dependency aruba (~> 0.2.1)
Building native extensions.  This could take a while...
Building native extensions.  This could take a while...

(::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::)

Thank you for installing cucumber-0.10.0.
Please be sure to read http://wiki.github.com/aslakhellesoy/cucumber/upgrading
for important information about this release. Happy cuking!

(::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::)

Building native extensions.  This could take a while...
**************************************************

  Thank you for installing rspec-core-2.3.1

  Please be sure to look at the upgrade instructions to see what might have
  changed since the last release:

  http://github.com/rspec/rspec-core/blob/master/Upgrade.markdown

**************************************************
Install development dependency sinatra (~> 1.1.0)? [Yn]  Y
Installing test dependency sinatra (~> 1.1.0)
Install development dependency bundler (~> 1.0.7)? [Yn]  Y
Installing test dependency bundler (~> 1.0.7)
Install development dependency cucumber (~> 0.9.4)? [Yn]  Y
Installing test dependency cucumber (~> 0.9.4)
Building native extensions.  This could take a while...

(::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::)

Thank you for installing cucumber-0.9.4.
Please be sure to read http://wiki.github.com/aslakhellesoy/cucumber/upgrading
for important information about this release. Happy cuking!

(::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::)

Install development dependency webmock (~> 1.6.0)? [Yn]  Y
Installing test dependency webmock (~> 1.6.0)
Install development dependency curb (~> 0.7.8)? [Yn]  Y
Installing test dependency curb (~> 0.7.8)
Building native extensions.  This could take a while...
Install development dependency patron (~> 0.4.6)? [Yn]  Y
Installing test dependency patron (~> 0.4.6)
Building native extensions.  This could take a while...
Install development dependency em-http-request (~> 0.2.7)? [Yn]  Y
Installing test dependency em-http-request (~> 0.2.7)
Building native extensions.  This could take a while...
Building native extensions.  This could take a while...
Install development dependency typhoeus (~> 0.2.1)? [Yn]  Y
Installing test dependency typhoeus (~> 0.2.1)
Building native extensions.  This could take a while...
rake aborted!
no such file to load -- bundler
/Users/myron/.rvm/gems/ruby-1.8.7-p330@gem-tester/gems/vcr-gem-testers-1.5.1/Rakefile:1
(See full trace by running task with --trace)
(in /Users/myron/.rvm/gems/ruby-1.8.7-p330@gem-tester/gems/vcr-gem-testers-1.5.1)

FWIW, here is what gem list shows has been installed:

➜  ~  gem list

*** LOCAL GEMS ***

addressable (2.2.4)
aruba (0.2.8)
builder (3.0.0, 2.1.2)
childprocess (0.1.6)
crack (0.1.8)
cucumber (0.10.0, 0.9.4)
cucumber_spinner (0.2.1)
curb (0.7.10)
diff-lcs (1.1.2)
em-http-request (0.2.15)
eventmachine (0.12.10)
fakeweb (1.3.0)
faraday (0.5.4)
ffi (0.6.3)
gherkin (2.3.3, 2.2.9)
httpclient (2.1.5.2)
json (1.4.6)
mime-types (1.16)
multipart-post (1.1.0)
patron (0.4.11)
rack (1.2.1, 1.1.0)
rake (0.8.7)
rspec (2.4.0, 2.3.0)
rspec-core (2.4.0, 2.3.1)
rspec-expectations (2.4.0, 2.3.0)
rspec-mocks (2.4.0, 2.3.0)
rtui (0.2.2)
rubygems-test (0.3.0)
shoulda (2.9.2)
sinatra (1.1.2)
term-ansicolor (1.0.5)
tilt (1.2.2)
timecop (0.3.5)
typhoeus (0.2.1)
vcr-gem-testers (1.5.1)
webmock (1.6.2)

As you can see, bundler is not installed in spite of the fact that it informed me that it was doing so.

Any ideas what the problem is?

BTW, I'm currently doing my testing using vcr-gem-testers rather than vcr so I can test how this works without actually making an official VCR release. My VCR branch with the code changes for this is here.

On the general topic of bundler support with this: I think it would be best for gem test to support a bundler Gemfile/Gemfile.lock when such a file is present. Lots of gems are using bundler for development these days. Bundler provide me with a guaranteed rubygems sandbox in a way that declaring dev dependencies doesn't. I can get close to the same thing by with regular development dependencies by declaring each and every dependent gem (and all of their dependencies, and their dependencies, ad infinitum) with the exact version number I want, but man...that would get so tedious and annoying. As I said, I have 17 direct development dependencies and over 40 total dependencies (counting dependencies of dependencies of dependencies). I don't really want to manage that by hand.

[rubiii]

a gemfile is definitely useful, but with gems you should never depend/check in a gemfile.lock for your gem. please read the following article: http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/

there's no "guaranteed sandbox" for gems through bundler. except when you're depending on exact versions in your gemspec that is. and even though your problem might be related (haven't looked at the full message yet), i might be wrong, but it sounds like you misunderstood bundler with rubygem dependecies.

ps. congrats on being a dad :)

[erikh]

Myron,

What version of rubygems are you using? 1.0.7 might not even install with RG 1.5.0, which is why I'm asking. If it's an issue that's not caused by that, I will happily look into it. Please file a new ticket if that's the case.

As for direct, please-rewrite-the-whole-thing bundler support (which is what you're asking for), I don't think that's a reasonable request under any circumstance. Feel free to write a bundler-based client to gem-testers.org or rework your test suites and gem dependencies to use bundler according to 'rake test'. It appears many have done this in their environments with no trouble according to the test results I've seen. I believe the twitter gem is using it among others.

Closing this ticket guys, sorry.

[myronmarston]

@erikh:

What version of rubygems are you using? 1.0.7 might not even install with RG 1.5.0, which is why I'm asking. If it's an issue that's not caused by that, I will happily look into it. Please file a new ticket if that's the case.

That might be the issue--thanks for the tip. I'll take a look.

As for direct, please-rewrite-the-whole-thing bundler support (which is what you're asking for), I don't think that's a reasonable request under any circumstance.

Maybe I'm missing something, but I don't see how it necessitates rewriting the whole thing; I was thinking it was as simple as a check for the presence of a Gemfile, and if it so, shelling out to bundle install to install dependencies rather than the existing method. That said, I may definitely be missing something, and if it does indeed require a large rewrite, then yes, my request is unreasonable.

or rework your test suites and gem dependencies to use bundler according to 'rake test'. It appears many have done this in their environments with no trouble according to the test results I've seen. I believe the twitter gem is using it among others.

I'm not totally sure what you mean, but my setup is pretty much like the twitter gem: I declare my dev dependencies in the gemspec, use bundler to install them, and require bundler in my Rakefile and spec_helper.rb so that it sets up the proper load paths for me.

@rubii:

a gemfile is definitely useful, but with gems you should never depend/check in a gemfile.lock for your gem. please read the following article: http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/

I read that blog post when it was posted, and I get what wycats is saying, but I don't completely agree. Sure, as a general rule, gems don't need a Gemfile.lock like applications do, and it's completely fine to ignore the Gemfile.lock. But it can still be useful in some situations, and unless I'm completely misunderstanding something, it doesn't cause any harm by having the Gemfile.lock checked in. I think it's useful because it means that for any contributors that clone the repository and run bundle install, they will get the exact same versions of the dev dependencies installed as I (as the gem author/maintainer) am using. This eliminates a whole class of potential development dependency version mis-match issues that could arise.

there's no "guaranteed sandbox" for gems through bundler.

Not when the gem is used by another gem or application, but in the test environment for the gem, bundler provides the same "guarenteed sandbox" that it does any other time you use it. Why wouldn't it?