search

rubygems / bundler

Manage your Ruby application's gem dependencies
https://bundler.io
MIT License
4.88k stars 1.99k forks source link

lockup resolving dependencies #1770

Closed bleything closed 11 years ago

bleything commented 12 years ago

Hello friends! I've got what I think is a bug, and Evan asked me to open a ticket. Consider this gemfile:

gem 'chef', '0.10.8'
gem 'knife-solo', '0.0.5'
gem 'knife-rackspace', '0.5.12'

bundle install will work fine here. But now add gem 'veewee' and try again. With bundler 1.0.22, I get no feedback, and if I Ctrl-C, I see this:

Fetching source index for http://bluebox.mirrors.rubygems.org/
^C
Quitting...
/Users/ben/.rvm/rubies/ree-1.8.7-2012.01/lib/ruby/site_ruby/1.8/rubygems/specification.rb:1348:in `initialize'
/Users/ben/.rvm/rubies/ree-1.8.7-2012.01/lib/ruby/site_ruby/1.8/rubygems/specification.rb:1341:in `each'
/Users/ben/.rvm/rubies/ree-1.8.7-2012.01/lib/ruby/site_ruby/1.8/rubygems/specification.rb:1341:in `initialize'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/remote_specification.rb:52:in `new'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/remote_specification.rb:52:in `method_missing'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:101:in `__dependencies'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:98:in `each'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:98:in `__dependencies'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:68:in `activate_platform'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:317:in `resolve_requirement'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:287:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:286:in `reverse_each'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:286:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:215:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:336:in `resolve_requirement'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:335:in `catch'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:335:in `resolve_requirement'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:287:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:286:in `reverse_each'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:286:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:336:in `resolve_requirement'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:335:in `catch'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:335:in `resolve_requirement'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:287:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:286:in `reverse_each'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:286:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:215:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:336:in `resolve_requirement'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:335:in `catch'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:335:in `resolve_requirement'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:287:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:286:in `reverse_each'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:286:in `resolve'
/Users/ben/.rvm/gems/ree-1.8.7-2012.01@global/gems/bundler-1.0.22/lib/bundler/resolver.rb:336:in `resolve_requirement'

There are hundreds more lines like that. With bundler 1.1.1, I can use --verbose, and I see this:

Fetching gem metadata from http://bluebox.mirrors.rubygems.org/
Query List: ["chef", "knife-solo", "knife-rackspace", "veewee"]
Query Gemcutter Dependency Endpoint API: chef knife-solo knife-rackspace veewee
Fetching from: http://bluebox.mirrors.rubygems.org/api/v1/dependencies?gems=chef,knife-solo,knife-rackspace,veewee
HTTP Redirection
Fetching from: http://rubygems.org/api/v1/dependencies?gems=chef,knife-solo,knife-rackspace,veewee?gems=chef,knife-solo,knife-rackspace,veewee
HTTP Success
Query List: ["uuidtools", "highline", "moneta", "erubis", "net-ssh-multi", "net-ssh", "treetop", "json", "bunny", "rest-client", "ohai", "mixlib-authentication", "mixlib-log", "mixlib-cli", "mixlib-config", "extlib", "amqp", "ruby-hmac", "stomp", "ruby-openid", "fog", "rspec", "cucumber", "progressbar", "thor", "popen4", "virtualbox", "vagrant", "CFPropertyList", "ruby-vnc", "ansi", "i18n"]
Query Gemcutter Dependency Endpoint API: uuidtools highline moneta erubis net-ssh-multi net-ssh treetop json bunny rest-client ohai mixlib-authentication mixlib-log mixlib-cli mixlib-config extlib amqp ruby-hmac stomp ruby-openid fog rspec cucumber progressbar thor popen4 virtualbox vagrant CFPropertyList ruby-vnc ansi i18n
Fetching from: http://bluebox.mirrors.rubygems.org/api/v1/dependencies?gems=uuidtools,highline,moneta,erubis,net-ssh-multi,net-ssh,treetop,json,bunny,rest-client,ohai,mixlib-authentication,mixlib-log,mixlib-cli,mixlib-config,extlib,amqp,ruby-hmac,stomp,ruby-openid,fog,rspec,cucumber,progressbar,thor,popen4,virtualbox,vagrant,CFPropertyList,ruby-vnc,ansi,i18n
HTTP Redirection
Fetching from: http://rubygems.org/api/v1/dependencies?gems=uuidtools,highline,moneta,erubis,net-ssh-multi,net-ssh,treetop,json,bunny,rest-client,ohai,mixlib-authentication,mixlib-log,mixlib-cli,mixlib-config,extlib,amqp,ruby-hmac,stomp,ruby-openid,fog,rspec,cucumber,progressbar,thor,popen4,virtualbox,vagrant,CFPropertyList,ruby-vnc,ansi,i18n?gems=uuidtools,highline,moneta,erubis,net-ssh-multi,net-ssh,treetop,json,bunny,rest-client,ohai,mixlib-authentication,mixlib-log,mixlib-cli,mixlib-config,extlib,amqp,ruby-hmac,stomp,ruby-openid,fog,rspec,cucumber,progressbar,thor,popen4,virtualbox,vagrant,CFPropertyList,ruby-vnc,ansi,i18n
HTTP Success
Query List: ["rake", "termios", "abstract", "net-ssh-gateway", "needle", "polyglot", "facets", "mime-types", "ipaddress", "yajl-ruby", "systemu", "english", "amq-protocol", "amq-client", "eventmachine", "hoe", "ruby-yadis", "nokogiri", "net-scp", "multi_json", "formatador", "excon", "builder", "named-parameters", "rspec-mocks", "rspec-expectations", "rspec-core", "diff-lcs", "term-ansicolor", "gherkin", "json_pure", "open4", "Platform", "ffi", "log4r", "childprocess", "archive-tar-minitar", "mario", "git-style-binaries", "tarruby", "libxml-ruby"]
Query Gemcutter Dependency Endpoint API: rake termios abstract net-ssh-gateway needle polyglot facets mime-types ipaddress yajl-ruby systemu english amq-protocol amq-client eventmachine hoe ruby-yadis nokogiri net-scp multi_json formatador excon builder named-parameters rspec-mocks rspec-expectations rspec-core diff-lcs term-ansicolor gherkin json_pure open4 Platform ffi log4r childprocess archive-tar-minitar mario git-style-binaries tarruby libxml-ruby
Fetching from: http://bluebox.mirrors.rubygems.org/api/v1/dependencies?gems=rake,termios,abstract,net-ssh-gateway,needle,polyglot,facets,mime-types,ipaddress,yajl-ruby,systemu,english,amq-protocol,amq-client,eventmachine,hoe,ruby-yadis,nokogiri,net-scp,multi_json,formatador,excon,builder,named-parameters,rspec-mocks,rspec-expectations,rspec-core,diff-lcs,term-ansicolor,gherkin,json_pure,open4,Platform,ffi,log4r,childprocess,archive-tar-minitar,mario,git-style-binaries,tarruby,libxml-ruby
HTTP Redirection
Fetching from: http://rubygems.org/api/v1/dependencies?gems=rake,termios,abstract,net-ssh-gateway,needle,polyglot,facets,mime-types,ipaddress,yajl-ruby,systemu,english,amq-protocol,amq-client,eventmachine,hoe,ruby-yadis,nokogiri,net-scp,multi_json,formatador,excon,builder,named-parameters,rspec-mocks,rspec-expectations,rspec-core,diff-lcs,term-ansicolor,gherkin,json_pure,open4,Platform,ffi,log4r,childprocess,archive-tar-minitar,mario,git-style-binaries,tarruby,libxml-ruby?gems=rake,termios,abstract,net-ssh-gateway,needle,polyglot,facets,mime-types,ipaddress,yajl-ruby,systemu,english,amq-protocol,amq-client,eventmachine,hoe,ruby-yadis,nokogiri,net-scp,multi_json,formatador,excon,builder,named-parameters,rspec-mocks,rspec-expectations,rspec-core,diff-lcs,term-ansicolor,gherkin,json_pure,open4,Platform,ffi,log4r,childprocess,archive-tar-minitar,mario,git-style-binaries,tarruby,libxml-ruby
HTTP Success
Query List: ["language", "rubyforge", "gemcutter", "RubyInline", "weakling", "trollop", "spruz", "shoulda"]
Query Gemcutter Dependency Endpoint API: language rubyforge gemcutter RubyInline weakling trollop spruz shoulda
Fetching from: http://bluebox.mirrors.rubygems.org/api/v1/dependencies?gems=language,rubyforge,gemcutter,RubyInline,weakling,trollop,spruz,shoulda
HTTP Redirection
Fetching from: http://rubygems.org/api/v1/dependencies?gems=language,rubyforge,gemcutter,RubyInline,weakling,trollop,spruz,shoulda?gems=language,rubyforge,gemcutter,RubyInline,weakling,trollop,spruz,shoulda
HTTP Success
Query List: ["ZenTest", "shoulda-matchers", "shoulda-context", "activesupport"]
Query Gemcutter Dependency Endpoint API: ZenTest shoulda-matchers shoulda-context activesupport
Fetching from: http://bluebox.mirrors.rubygems.org/api/v1/dependencies?gems=ZenTest,shoulda-matchers,shoulda-context,activesupport
HTTP Redirection
Fetching from: http://rubygems.org/api/v1/dependencies?gems=ZenTest,shoulda-matchers,shoulda-context,activesupport?gems=ZenTest,shoulda-matchers,shoulda-context,activesupport
HTTP Success
Query List: ["memcache-client", "tzinfo"]
Query Gemcutter Dependency Endpoint API: memcache-client tzinfo
Fetching from: http://bluebox.mirrors.rubygems.org/api/v1/dependencies?gems=memcache-client,tzinfo
HTTP Redirection
Fetching from: http://rubygems.org/api/v1/dependencies?gems=memcache-client,tzinfo?gems=memcache-client,tzinfo
HTTP Success
Query List: []
Unmet Dependencies:

... it hangs at that last line for longer than I have patience to wait.

mconigliaro commented 12 years ago

Good, it's not just me.

indirect commented 12 years ago

There is a magic "even more verbose" option! Run DEBUG_RESOLVER=1 bundle install to see exactly what is happening inside the resolver. Chances are that this will help you figure out exactly where it is getting stuck. If there is a way to keep the resolver from going off into the weeds, that's where the key will lie.

ghost commented 12 years ago

This looks odd ―

Fetching from: http://rubygems.org/api/v1/dependencies?gems=memcache-client,tzinfo?gems=memcache-client,tzinfo

Specifically the ?gems=memcache-client,tzinfo?gems=memcache-client,tzinfo

bryanstearns commented 12 years ago

veewee seems to have conflicting requirements on its own, so a simpler way to see this is:

git clone https://github.com/jedi4ever/veewee
cd veewee
git checkout 764f56ba
DEBUG_RESOLVER=1 bundle install

(if you're using RVM, its rvmrc wants a 1.9.2 ruby. Yes, veewee needs to fix its requirements, but Bundler shouldn't hang trying to resolve bad ones :-) )

indirect commented 12 years ago

What do you mean by "hang"? Does it stop trying to resolve? Or does it take a long time to resolve? There are known cases where resolving can take a very long time if the graph is extremely complicated.

bryanstearns commented 12 years ago

Thanks André; I meant "hang" in that I gave up after five minutes (without the extra logging); it quickly pinned one CPU of my core i7 box. Doesn't seem to be leaking memory or doing any I/O (no surprise there). Turning the extra logging on produced 14Mb of output in the first 10 seconds, which is why I didn't post it; I did look through it a little, only to see that it seemed to be cycling through acceptable vagrant versions.

Looking more closely, it is permuting acceptable versions of several gems, and if I let it run (without logging), it does eventually finish successfully (after 6 minutes 31 seconds).

I'm guessing that Ben's original problem above is the result of the difficult search to satisfy veewee, complicated by the other dependencies in his Gemfile. I set up his gemfile (with Bundler 1.1.3) and let it run; it still hasn't finished after 15 hours.

trinary commented 11 years ago

Triaging this: just tried to reproduce the steps above, current bundler 1.3.0 seems to resolve dependencies (installed vagrant 1.0.6). I think this can be closed.

indirect commented 11 years ago

Great, thanks for reporting back!