Bundle resolving dependencies slowly

[TangMonk]

I want to install this gem, only a few dependencies gem. But when resolving dependencies, bundle Through all of my gem!

This is my forked repo git://github.com/TangMonk/geokit.git I add gem 'geokit', github: 'TangMonk/geokit', branch: 'master' to my Gemfile. Then I type command $ bundle install

Fetching git://github.com/TangMonk/geokit.git
remote: Counting objects: 2222, done.
remote: Compressing objects: 100% (1039/1039), done.
remote: Total 2222 (delta 1275), reused 2099 (delta 1152)
Receiving objects: 100% (2222/2222), 12.28 MiB | 352.00 KiB/s, done.
Resolving deltas: 100% (1275/1275), done.
Fetching gem metadata from http://rubygems.org/........
Fetching gem metadata from http://rubygems.org/..
Resolving dependencies...
Enter your password to install the bundled RubyGems to your system:
Using rake (10.1.0)
Using i18n (0.6.9)
Using minitest (4.7.5)
Using multi_json (1.8.2)
Using atomic (1.1.14)
Using thread_safe (0.1.3)
Using tzinfo (0.3.38)
Using activesupport (4.0.1)
Using builder (3.1.4)
Using erubis (2.7.0)
Using rack (1.5.2)
Using rack-test (0.6.2)
Using actionpack (4.0.1)
Using mime-types (1.25.1)
Using polyglot (0.3.3)
Using treetop (1.4.15)
Using mail (2.5.4)
Using actionmailer (4.0.1)
Using activemodel (4.0.1)
Using activerecord-deprecated_finders (1.0.3)
Using arel (4.0.1)
Using activerecord (4.0.1)
Using json (1.8.1)
Using multi_xml (0.5.5)
Using httparty (0.12.0)
Using baidu_push (0.0.2)
Using bcrypt-ruby (3.1.2)
Using bundler (1.3.5)
Using diff-lcs (1.2.5)
Using geokit (1.8.2) from git://github.com/TangMonk/geokit.git (at master)
Using hike (1.2.3)
Using mysql2 (0.3.14)
Using thor (0.18.1)
Using railties (4.0.1)
Using tilt (1.4.1)
Using sprockets (2.10.1)
Using sprockets-rails (2.0.1)
Using rails (4.0.1)
Using rspec-support (3.0.0.beta1)
Using rspec-expectations (3.0.0.beta1)
Using rspec-collection_matchers (0.0.2)
Using rspec-core (3.0.0.beta1)
Using rspec-mocks (3.0.0.beta1)
Using rspec-rails (3.0.0.beta1)

Is it necessary?

[Who828]

Hello there,

Can you please supply all of the information requested in ISSUES so that we can try to diagnose the problem?

[TangMonk]

@Who828 , Hi , I update my question. Thanks

[Who828]

@TangMonk Can you show me your

bundle env

output for the project you are currently running bundle install?

[TangMonk]

@Who828

Bundler 1.3.5
Ruby 2.0.0 (2013-06-27 patchlevel 247) [universal.x86_64-darwin13]
Rubygems 2.1.11
rvm 1.23.14 (stable)
GEM_HOME

Gemfile
source 'http://rubygems.org'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '4.0.1'

# Use mysql as the database for Active Record
gem 'mysql2'

gem 'bcrypt-ruby'

gem 'baidu_push'

gem 'geokit', github: 'TangMonk/geokit', branch: 'master'

group :development, :test do
    gem 'rspec-rails'
    gem 'rspec-collection_matchers'
end

Gemfile.lock
GIT
  remote: git://github.com/TangMonk/geokit.git
  revision: b8486cfb4501bc5e86b2f9f37d9b3bd2825e10dc
  branch: master
  specs:
    geokit (1.8.2)
      multi_json (>= 1.3.2)

GEM
  remote: http://rubygems.org/
  specs:
    actionmailer (4.0.1)
      actionpack (= 4.0.1)
      mail (~> 2.5.4)
    actionpack (4.0.1)
      activesupport (= 4.0.1)
      builder (~> 3.1.0)
      erubis (~> 2.7.0)
      rack (~> 1.5.2)
      rack-test (~> 0.6.2)
    activemodel (4.0.1)
      activesupport (= 4.0.1)
      builder (~> 3.1.0)
    activerecord (4.0.1)
      activemodel (= 4.0.1)
      activerecord-deprecated_finders (~> 1.0.2)
      activesupport (= 4.0.1)
      arel (~> 4.0.0)
    activerecord-deprecated_finders (1.0.3)
    activesupport (4.0.1)
      i18n (~> 0.6, >= 0.6.4)
      minitest (~> 4.2)
      multi_json (~> 1.3)
      thread_safe (~> 0.1)
      tzinfo (~> 0.3.37)
    arel (4.0.1)
    atomic (1.1.14)
    baidu_push (0.0.2)
      httparty (>= 0.11.0)
    bcrypt-ruby (3.1.2)
    builder (3.1.4)
    diff-lcs (1.2.5)
    erubis (2.7.0)
    hike (1.2.3)
    httparty (0.12.0)
      json (~> 1.8)
      multi_xml (>= 0.5.2)
    i18n (0.6.9)
    json (1.8.1)
    mail (2.5.4)
      mime-types (~> 1.16)
      treetop (~> 1.4.8)
    mime-types (1.25.1)
    minitest (4.7.5)
    multi_json (1.8.2)
    multi_xml (0.5.5)
    mysql2 (0.3.14)
    polyglot (0.3.3)
    rack (1.5.2)
    rack-test (0.6.2)
      rack (>= 1.0)
    rails (4.0.1)
      actionmailer (= 4.0.1)
      actionpack (= 4.0.1)
      activerecord (= 4.0.1)
      activesupport (= 4.0.1)
      bundler (>= 1.3.0, < 2.0)
      railties (= 4.0.1)
      sprockets-rails (~> 2.0.0)
    railties (4.0.1)
      actionpack (= 4.0.1)
      activesupport (= 4.0.1)
      rake (>= 0.8.7)
      thor (>= 0.18.1, < 2.0)
    rake (10.1.0)
    rspec-collection_matchers (0.0.2)
      rspec-expectations (>= 2.99.0.beta1)
    rspec-core (3.0.0.beta1)
      rspec-support (= 3.0.0.beta1)
    rspec-expectations (3.0.0.beta1)
      diff-lcs (>= 1.1.3, < 2.0)
      rspec-support (= 3.0.0.beta1)
    rspec-mocks (3.0.0.beta1)
      rspec-support (= 3.0.0.beta1)
    rspec-rails (3.0.0.beta1)
      actionpack (>= 3.0)
      activemodel (>= 3.0)
      activesupport (>= 3.0)
      railties (>= 3.0)
      rspec-collection_matchers
      rspec-core (= 3.0.0.beta1)
      rspec-expectations (= 3.0.0.beta1)
      rspec-mocks (= 3.0.0.beta1)
    rspec-support (3.0.0.beta1)
    sprockets (2.10.1)
      hike (~> 1.2)
      multi_json (~> 1.0)
      rack (~> 1.0)
      tilt (~> 1.1, != 1.3.0)
    sprockets-rails (2.0.1)
      actionpack (>= 3.0)
      activesupport (>= 3.0)
      sprockets (~> 2.8)
    thor (0.18.1)
    thread_safe (0.1.3)
      atomic
    tilt (1.4.1)
    treetop (1.4.15)
      polyglot
      polyglot (>= 0.3.1)
    tzinfo (0.3.38)

PLATFORMS
  ruby

DEPENDENCIES
  baidu_push
  bcrypt-ruby
  geokit!
  mysql2
  rails (= 4.0.1)
  rspec-collection_matchers
  rspec-rails

[Who828]

@TangMonk Thank you for the information.

1) Yes its very important to resolve all the dependencies when you add a new gem because that particular gem might have a requirement which would conflict with the current set of requirements (in this case it has a requirement on multi-json which is common with lot of other gems). That's why to ensure things are working smoothly, we do resolve all the dependencies again. (that means it makes a call to rubygems server to get all its requirements, different versions etc).

2) In your case the resolution of dependencies is fairly quick

Resolving dependencies...

It's just takes 3 iterations to resolve all of your gems.

3) Most of the time is just spent in making the call to rubygems and loading all the gems again. (you can check by calling bundle install --verbose)

Also, @indirect is actively working on a new index format which will speedup the network calls significantly (the 3rd point).

[TangMonk]

@Who828 ,Thanks, I love your logo :)

BTW, I used the Node.js, I thinks The NPM is better than bundle. Why bundle not to learn from NPM?

[Who828]

@TangMonk

Always remember when it comes to CS, everything is a tradeoff.

the way NPM handles its requirements is not possible in Ruby. Basically it installs all the versions of dependencies required by the packages. So if package A requires X-0.02 and package B requires X-0.01. Both of them can use different version of X because the require is specific to that namespace, which is not the case in Ruby. (require imports it to a global namespace AFAIK).

Having said that, I don't think NPM approach is absolutely the best one. It has its disadvantages. For example, debugging could be a nightmare in a mature project because different packages would have different versions of libs. Also, the project becomes a bloat with so many different versions of dependencies.

Our approach also has its downsides but we are always improving them, for example we are trying to improve our index format which would speedup bundle install significantly (since it will reduce the network calls we make). We're also improving our dependency algorithm so it resolves better and faster. Also we just added support for parallel gem installation in Bundler 1.5 which has improved bundle install performance by 40-60%.

So all in all, we're are very proud of our work in Bundler as we continue to improve it :)