Get Deadlock when trying parallel gem installing with bundler-1.14.5, ruby-2.4.0p0

[krhougs]

Error Report

I ran the command /usr/local/bin/bundle install --path .bundle --jobs 1 --retry 3

It seemed to have performances a Deadlock with bundler 1.14.5, while bundler 1.13.x performances the true behavior.

Backtrace

fatal: No live threads left. Deadlock?
2 threads, 2 sleeps current:0x00000001beeff0 main thread:0x00000000a9e6c0
* #<Thread:0x00000000acf800 sleep_forever>
   rb_thread_t:0x00000000a9e6c0 native:0x007f4899e4f700 int:0
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:43:in `pop'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:43:in `deq'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer/parallel_installer.rb:131:in `process_specs'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer/parallel_installer.rb:102:in `call'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer/parallel_installer.rb:77:in `call'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer.rb:199:in `install_in_parallel'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer.rb:166:in `install'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer.rb:80:in `run'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer.rb:24:in `install'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli/install.rb:70:in `run'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli.rb:181:in `block in install'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/settings.rb:99:in `temporary'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli.rb:180:in `install'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli.rb:20:in `dispatch'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli.rb:11:in `start'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/exe/bundle:32:in `block in <top (required)>'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/exe/bundle:24:in `<top (required)>'
   /usr/local/bin/bundle:22:in `load'
   /usr/local/bin/bundle:22:in `<main>'
* #<Thread:0x000000030a8f68@Parallel Installer Worker #0@/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:88 sleep_forever>
   rb_thread_t:0x00000001beeff0 native:0x007f48949b6700 int:0
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:56:in `pop'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:56:in `block in process_queue'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:55:in `loop'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:55:in `process_queue'
   /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:88:in `block (2 levels) in create_threads'

  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:43:in `pop'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/worker.rb:43:in `deq'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer/parallel_installer.rb:131:in `process_specs'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer/parallel_installer.rb:102:in `call'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer/parallel_installer.rb:77:in `call'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer.rb:199:in `install_in_parallel'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer.rb:166:in `install'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer.rb:80:in `run'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/installer.rb:24:in `install'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli/install.rb:70:in `run'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli.rb:181:in `block in install'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/settings.rb:99:in `temporary'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli.rb:180:in `install'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli.rb:20:in `dispatch'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/cli.rb:11:in `start'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/exe/bundle:32:in `block in <top (required)>'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
  /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.5/exe/bundle:24:in `<top (required)>'
  /usr/local/bin/bundle:22:in `load'
  /usr/local/bin/bundle:22:in `<main>'

Environment

Bundler   1.14.5
Rubygems  2.6.10
Ruby      2.4.0p0 (2016-12-24 revision 57164) [x86_64-linux]
GEM_HOME  /builds/stormcloudcode/fengbao-cloud-backend/.bundle/ruby/2.4.0
GEM_PATH  
Git       2.1.4
Platform  x86_64-linux
OpenSSL   OpenSSL 1.0.1t  3 May 2016

Bundler settings

mirror.https://rubygems.org/
  Set for the current user (/root/.bundle/config): "https://gems.ruby-china.org"
retry
  Set for your local app (/usr/local/bundle/config): 3
path
  Set for your local app (/usr/local/bundle/config): ".bundle"
  Set via BUNDLE_PATH: "/usr/local/bundle"
jobs
  Set for your local app (/usr/local/bundle/config): 1
disable_shared_gems
  Set for your local app (/usr/local/bundle/config): true
silence_root_warning
  Set via BUNDLE_SILENCE_ROOT_WARNING: "1"
app_config
  Set via BUNDLE_APP_CONFIG: "/usr/local/bundle"
bin
  Set via BUNDLE_BIN: "/usr/local/bundle/bin"

My Gemfile goes:

source 'https://gems.ruby-china.org'

git_source(:github) do |repo_name|
  repo_name = "#{repo_name}/#{repo_name}" unless repo_name.include?("/")
  "https://github.com/#{repo_name}.git"
end

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 5.0.1'
# Use postgresql as the database for Active Record
gem 'pg', '~> 0.18'
# Use Puma as the app server
gem 'puma', '~> 3.0'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 2.5'
# Use Redis adapter to run Action Cable in production
gem 'redis', '~> 3.0'
# Use ActiveModel has_secure_password
gem 'bcrypt', '~> 3.1.7'
gem 'ancestry'
gem 'sanitize'
gem 'pg_search'
gem 'action-store'

# Use Capistrano for deployment
# gem 'capistrano-rails', group: :development

# Use Rack CORS for handling Cross-Origin Resource Sharing (CORS), making cross-origin AJAX possible
gem 'rack-cors'
gem 'rack-attack'
gem 'qiniu', '~> 6.8.1'

group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platform: :mri
  # Use RSpec for specs
  gem 'rspec-rails', '>= 3.5.0'
  # Use Factory Girl for generating random test data
  gem 'factory_girl_rails'

  gem 'rubocop', require: false
  gem 'apipie-rails'
end

group :development do
  gem 'listen', '~> 3.0.5'
  # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
  gem 'spring'
  gem 'spring-watcher-listen', '~> 2.0.0'
end

gem 'capistrano', require: false
gem 'capistrano-rails', require: false
gem 'capistrano-bundler', require: false
gem 'capistrano3-puma', require: false
gem 'capistrano-rvm', require: false
gem 'capistrano-rails-db', require: false

group :test do
  gem 'database_cleaner', :require => false
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

[segiddins]

This does not deadlock for me

[colby-swandale]

ping @oranzhang

[krhougs]

I find this in both production docker container and development macOS when I enabled Parallel Installer.

[colby-swandale]

I installed your bundle as close to your env as i could get and cannot reproduce your problem, can you share any more info?

[krhougs]

It seems to be fixed in #5480, and I can neither reproduce with these dependencies updated. Problem solved with bundler 1.14.6.