asdf does not seem to be picking up on ruby gem executables in the GEM_PATH

[jedschneider]

I've been using asdf for at least 4-5 years across languages and this is the first time I've experienced any unexpected behavior. While this report is specific to rspec, it has been observed for other bin executables like foreman.

version info:

asdf version 0.14.0
ruby version : 3.2.1
bundler version 2.4.21
rspec-core 3.12.2

asdf install via homebrew, zsh shell.

.zshrc hook

. /opt/homebrew/opt/asdf/libexec/asdf.sh

asdf info

% asdf info
OS:
Darwin wingmate.local 23.2.0 Darwin Kernel Version 23.2.0: Wed Nov 15 21:53:34 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T8103 arm64

SHELL:
zsh 5.9 (arm-apple-darwin22.1.0)

BASH VERSION:
3.2.57(1)-release

ASDF VERSION:
v0.14.0

ASDF INTERNAL VARIABLES:
ASDF_DEFAULT_TOOL_VERSIONS_FILENAME=.tool-versions
ASDF_DATA_DIR=/Users/jed/.asdf
ASDF_DIR=/opt/homebrew/opt/asdf/libexec
ASDF_CONFIG_FILE=/Users/jed/.asdfrc

ASDF INSTALLED PLUGINS:
python                       https://github.com/danhper/asdf-python.git master 5e277e2
ruby                         https://github.com/asdf-vm/asdf-ruby.git master 7a22142
terraform                    https://github.com/asdf-community/asdf-hashicorp.git master 197e3ec

scenario

I recently began observing that gems that should exist in the bundle are not resolving on the $PATH. As an example, I have a rails app that is currently running ruby 3.2.1. I followed the instructions from the asdf readme to fully remove and then reinstall asdf via homebrew, add the ruby plugin, and install ruby 3.2.1. I was able to cd into my rails app, and bundle the app successfully. Then, when running rspec within the context of the bundle, I received the following output:

% bundle exec rspec spec
bundler: command not found: rspec
Install missing gem executables with `bundle install`

ruby options in env

% env | grep -i ruby
RUBYOPT=-W:no-deprecated -W:no-experimental

% ruby -v
ruby 3.2.1 (2023-02-08 revision 31819e82c8) [arm64-darwin23]
jed@wingmate src % asdf list ruby
 *3.2.1

the command does not appear in my path:

% command -V rspec #=> 🚫 

bundler info:

% bundle show rspec-core
/Users/jed/.gem/ruby/3.2.0/gems/rspec-core-3.12.2

% gem which bundler
/Users/jed/.asdf/installs/ruby/3.2.1/lib/ruby/3.2.0/bundler.rb

% bundle config
Settings are listed in order of priority. The top value will be used.
deployment
Set for the current user (/Users/jed/.bundle/config): false

force_ruby_platform
Set for the current user (/Users/jed/.bundle/config): true

gem.changelog
Set for the current user (/Users/jed/.bundle/config): true

gem.ci
Set for the current user (/Users/jed/.bundle/config): "none"

gem.coc
Set for the current user (/Users/jed/.bundle/config): false

gem.linter
Set for the current user (/Users/jed/.bundle/config): "none"

gem.mit
Set for the current user (/Users/jed/.bundle/config): true

gem.test
Set for the current user (/Users/jed/.bundle/config): "rspec"

jobs
Set for the current user (/Users/jed/.bundle/config): 7

silence_root_warning
Set for the current user (/Users/jed/.bundle/config): true

% bundle info rspec-core | grep Path
Path: /Users/jed/.gem/ruby/3.2.0/gems/rspec-core-3.12.2

bundler shim:

% cat /Users/jed/.asdf/shims/bundle
#!/usr/bin/env bash
# asdf-plugin: ruby 3.2.1
exec /opt/homebrew/opt/asdf/libexec/bin/asdf exec "bundle" "$@" # asdf_allow: ' asdf '

identifying the bundler and gem exe paths:

% bundle exec gem contents rspec-core | grep exe
/Users/jed/.gem/ruby/3.2.0/gems/rspec-core-3.12.2/exe/rspec

spec runs when prefixing the whole path:

% /Users/jed/.gem/ruby/3.2.0/gems/rspec-core-3.12.2/exe/rspec spec/some_spec.rb # => 👍 

and there is also a bin exe that when fully pathed works just fine

/Users/jed/.gem/ruby/3.2.0/bin/rspec spec/my_spec.rb # => 👍 

temporary solution

If I globally install the gem to the ruby version, I get a shim for the executable that will work:

% ls ~/.asdf/shims | grep rspec #=> 🚫 

% gem install rspec-core

% cat /Users/jed/.asdf/shims/rspec
#!/usr/bin/env bash
# asdf-plugin: ruby 3.2.1
exec /opt/homebrew/opt/asdf/libexec/bin/asdf exec "rspec" "$@" # asdf_allow: ' asdf '

% bundle exec rspec spec/my_spec.rb #=> 👍 

I was able to get to this temporary solution by groking enough of the reshim.sh in asdf to understand how it would work. The question is how the ruby asdf plugin should work in order to dynamically adjust the path to pick up on the bin executables, and if there has been something that has changed recently that would cause a change in this behavior for either this specific ruby version, the specific bundler version, or asdf as a whole, as I do not know where to target any further investigation.

Thanks in advance!

[monfresh]

This seems to be the issue:

% bundle info rspec-core | grep Path
Path: /Users/jed/.gem/ruby/3.2.0/gems/rspec-core-3.12.2

It looks like you have rspec-core installed in 3.2.0, but not in 3.2.1. Are you sure your project's .ruby-version and Gemfile are specifying 3.2.1? Perhaps you haven't yet run bundle install while using Ruby 3.2.1?

[jedschneider]

Thank you for the quick reply.

I thought similar at first, but it was brought to my attention that this expected behavior because 3.2 is the ABI version for 3.2.1? That seems logical to me, but I don't know how to verify that easily. Some other supporting evidence:

I generated a binstub with bundler bundle binstub rspec and then rspec was able to be run with the binstub.

I only have one ruby installed

bundle install ran cleanly and should have installed that gem to that directory but my .gem dir contains no minor versions:

% ls /Users/jed/.gem/ruby/
2.6.0   3.2.0   3.3.0

.tool_versions

terraform 1.3.7
python 3.10.11
ruby 3.2.1

head of Gemfile

source "https://rubygems.org"
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby "~> 3.2.1"

Gemfile.lock exerts.

PLATFORMS
  arm64-darwin-21
  ruby
  x86_64-linux

RUBY VERSION
   ruby 3.2.1p31

BUNDLED WITH
   2.4.21

No legacy .ruby_version file in the project or in the parent file hierarchy, at least up to $HOME.

[jedschneider]

One other item of note, this did seem to start after a brew update/upgrade and an associated ruby plugin update sadly I did not track the changes too closely but it was probably related to updating the ruby versions available so I could get the 3.3.0 releases and to ensure my statically linked libs were up to date to build the new ruby version.

[jedschneider]

Looking at #378 I'm wondering if this is an issue because rspec-core does not use the bin directory for executables.

EDIT: there is a bindir setting for rspec-core, so still searching for the way in which asdf should dynamically switch to gem executables per ruby version.

[monfresh]

Right, I'm used to chruby, and I have minor versions in my ~/.gem/ruby folder. But asdf behaves differently. Out of curiosity, I switched to asdf (which my Ruby on Mac product makes super easy; shameless plug), and I was not able to reproduce your issue. Here's what I did:

rom config set manager asdf
rom install ruby 3.2.1
cd ~/projects
mkdir rails-test
cd rails-test
rails new .

This used 3.2.3 by default, so then I updated the ~/.ruby-version and Gemfile to point to Ruby 3.2.1, and then:

bundle install
bundle add rspec-core

command -v rspec
/Users/moncefb/.asdf/shims/rspec

bundle exec rspec
No examples found.

Finished in 0.00025 seconds (files took 0.20698 seconds to load)
0 examples, 0 failures

bundle info rspec-core | grep Path
Path: /Users/moncefb/.asdf/installs/ruby/3.2.1/lib/ruby/gems/3.2.0/gems/rspec-core-3.12.2

So, it sounds like something in your dev setup is putting the gem path ~/.gem/ruby before the asdf path. When I run gem env, I get this order:

- GEM PATHS:
     - /Users/moncefb/.asdf/installs/ruby/3.2.1/lib/ruby/gems/3.2.0
     - /Users/moncefb/.gem/ruby/3.2.0

and the installation directory also points to asdf:

RubyGems Environment:
  - RUBYGEMS VERSION: 3.5.5
  - RUBY VERSION: 3.2.1 (2023-02-08 patchlevel 31) [arm64-darwin23]
  - INSTALLATION DIRECTORY: /Users/moncefb/.asdf/installs/ruby/3.2.1/lib/ruby/gems/3.2.0

Here's what I would try:

• Check your .zshrc, .zprofile, .zlogin for anything that might affect your PATH • Update to the latest bundler and Rubygems • Backup and remove (or temporarily rename) your ~/.bundle/config

[monfresh]

I should have specified that it's probably the GEM_PATH and/or GEM_HOME env vars that are possibly being set somewhere. You can check with env | grep GEM. Also, are you perhaps specifying --user-install in your ~/.gemrc?

[gsmetal]

I've run into the similar issue: binaries from gems installed from git repo in Gemfile do not work. Looks like an old issue #194, but I did not understand the solution from there.