Change in behaviour of Bundler.definition.specs_for and BUNDLE_WITHOUT

[kaoru]

This is causing issues with AWS Elastic Beanstalk instances running Amazon Linux + Ruby + Puma. See my comment https://github.com/bundler/bundler/issues/5861#issuecomment-327929004 for more details on that.

This might also be relevant to https://github.com/bundler/bundler/issues/5865 if the problem there is BUNDLE_WITHOUT.

I've managed to reproduce the problem with a very small environment, which should isolate the bug and help to fix it:

Directory

drwxr-xr-x 2 alex alex 4096 Sep  8 17:12 .bundle/
-rw-r--r-- 1 alex alex  167 Sep  8 17:11 Gemfile
-rwxr-xr-x 1 alex alex  371 Sep  8 17:16 test.rb*

.bundle/config

BUNDLE_WITHOUT: test:development
BUNDLE_DISABLE_SHARED_GEMS: true

Gemfile

source 'https://rubygems.org'

gem 'public_suffix'

group :development do
  gem 'koala'
end

group :test do
  gem 'chronic'
end

group :production do
  gem 'haml'
end

test.rb

#!/usr/bin/env ruby

require 'bundler'

puts "Bundler version: #{Bundler::VERSION}"

groups = Bundler.definition.groups

begin
  Bundler.definition.resolve_with_cache!
rescue Bundler::GemNotFound
  puts "RESOLVING REMOTELY"
  Bundler.definition(true).resolve_remotely!
end

def count_specs_for_group(group)
  Bundler.definition.specs_for([group]).count
rescue => e
  "failed: #{e.class} #{e.message}"
end

groups.each do |group|
  puts "#{group}: #{count_specs_for_group(group)}"
end

Output with Bundler 1.15.4 (BAD)

Bundler version: 1.15.4
default: 2
development: failed: RuntimeError Unable to find a spec satisfying koala in the set. Perhaps the lockfile is corrupted?
test: failed: RuntimeError Unable to find a spec satisfying chronic in the set. Perhaps the lockfile is corrupted?
production: 4

Output with Bundler 1.12.1 (GOOD)

Bundler version: 1.12.1
RESOLVING REMOTELY
default: 2
development: 1
test: 1
production: 4

[segiddins]

Definition#specs_for is considered a private API and its use outside of bundler itself is unsupported

[kaoru]

Somebody tell that to AWS ElasticBeanstalk :disappointed: Is there a simple public-API-only way to write the /opt/elasticbeanstalk/support/scripts/check-for-gem.rb script that they use?

#!/usr/bin/env /opt/rubies/ruby-current/bin/ruby
#==============================================================================
# Copyright 2014 Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Amazon Software License (the "License"). You may not use
# this file except in compliance with the License. A copy of the License is
# located at
#
#       https://aws.amazon.com/asl/
#
# or in the "license" file accompanying this file. This file is distributed on
# an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
# implied. See the License for the specific language governing permissions
# and limitations under the License.
#==============================================================================
require 'bundler'

Dir.chdir '/var/app/ondeck'
groups = Bundler.definition.groups

begin
    Bundler.definition.resolve_with_cache!
rescue Bundler::GemNotFound
    Bundler.definition(true).resolve_remotely!
end

gems = Bundler.definition.specs_for(groups).collect(&:name)
exit 0 if gems.include? ARGV[0]

Either way thanks for your response @segiddins!

[segiddins]

Not that I can think of at the moment

[kaoru]

Shame. Thanks for your help @segiddins, feel free to close this if it's not providing any benefit.

Will be interesting to see if Amazon Linux 2017.03 2.4.5 solves it somehow.

[philayres]

I'm experiencing this too. @kaoru did you find a workaround? [EDIT] I found that removing the .ebextensions/abc.config file I had put in place to facilitate deployment hid the error. I could at least redeploy my app. Perhaps it is due to AWS calling the rake statement I had placed in there that exposes this error. Anyway, may be useful for somebody else stumbling across this error.

[kaoru]

I'm afraid not @philayres, for now I've just downgraded to Amazon Linux 2017.03 2.4.0 :-( Hoping Amazon releases a new version soon.

Sorry you're having the same issue, but at the same time somewhat glad to hear it's not just me :sweat_smile:

[lovesmall]

Hi @kaoru, this issue has been resolved in the newly released 2.5.0.

Release notes: https://aws.amazon.com/releasenotes/AWS-Elastic-Beanstalk/1578013167042124

[kaoru]

Fantastic news, thanks for the heads up @lovesmall :grinning: