gem download fails with EPERM

[ksperling]

karsten@qualifier:~$ brew gem install papertrail
==> Using the sandbox
==> Fetching papertrail from gem source
ERROR:  While executing gem ... (Errno::EPERM)
    Operation not permitted - /Users/karsten/.gem/specs/api.rubygems.org%443/specs.4.8
==> /usr/bin/gem install /Users/karsten/Library/Caches/Homebrew/papertrail-0.10.3.gem --no-ri --no-rdoc --no-wrapper --no-user-install --install-dir /usr/local/
Last 15 lines from /Users/karsten/Library/Logs/Homebrew/gem-papertrail/01.gem:
2017-08-03 00:07:18 +1200

/usr/bin/gem
install
/Users/karsten/Library/Caches/Homebrew/papertrail-0.10.3.gem
--no-ri
--no-rdoc
--no-wrapper
--no-user-install
--install-dir
/usr/local/Cellar/gem-papertrail/0.10.3
--bindir
/usr/local/Cellar/gem-papertrail/0.10.3/bin

ERROR:  Could not find a valid gem '/Users/karsten/Library/Caches/Homebrew/papertrail-0.10.3.gem' (>= 0) in any repository

Looking at the source, it seems brew-gem just runs gem fetch ... from the homebrew cache directory (however it should check the exit code and not continue if the download failed)

I haven't been able to figure out why gem fetch even fails in the first place though. The permissions of .gem/specs/api.rubygems.org%443 look fine. When I cd into the homebrew cache and run fetch manually it works:

karsten@qualifier:~/Library/Caches/Homebrew$ gem fetch papertrail --version 0.10.3
Fetching: papertrail-0.10.3.gem (100%)
Downloaded papertrail-0.10.3

After this brew gem install succeeds -- it attempts the download again (which fails again), but then proceeds to install the gem:

karsten@qualifier:~/Library/Caches/Homebrew$ brew gem install papertrail
==> Using the sandbox
==> Fetching papertrail from gem source
ERROR:  While executing gem ... (Errno::EPERM)
    Operation not permitted - /Users/karsten/.gem/specs/api.rubygems.org%443/specs.4.8
Warning: Cannot verify integrity of papertrail-0.10.3.gem
A checksum was not provided for this resource
For your reference the SHA256 is: a29916a203a7960b25191f6b0bca671580285273a71f6de9455c08997fe9c5de
==> /usr/bin/gem install /Users/karsten/Library/Caches/Homebrew/papertrail-0.10.3.gem --no-ri --no-rdoc --no-wrapper --no-user-install --install-dir /usr/local/
🍺  /usr/local/Cellar/gem-papertrail/0.10.3: 157 files, 513.3KB, built in 4 seconds

karsten@qualifier:~$ gem --version
karsten@qualifier:~$ which gem
/usr/bin/gem
karsten@qualifier:~$ gem --version
2.6.12
karsten@qualifier:~$ which ruby
/usr/bin/ruby
karsten@qualifier:~$ ruby --version
ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin15]
karsten@qualifier:~$ brew --version
Homebrew 1.3.0-15-g2ad03b8
Homebrew/homebrew-core (git revision 4b5dd; last commit 2017-08-02)
karsten@qualifier:~$ brew info brew-gem
brew-gem: stable 0.8.1, HEAD
Install rubygems as homebrew formulae
https://github.com/sportngin/brew-gem
/usr/local/Cellar/brew-gem/0.8.1 (10 files, 11.9KB) *
  Built from source on 2017-08-03 at 00:24:08
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/brew-gem.rb

[nicksieger]

We have seen this happening too -- perhaps there are changes to or issues with rubygems.org?

[ksperling]

Possible, but somewhat unlikely given that the download failed when run from brew-gem both before and after the successful manual download. Is it possible it's some kind of environment issue like a different ruby or gem binary being run via brew-gem?

[nayzak]

Both system and homebrew gems are work fine when executing fetch command manually

 ~/Library/Caches/Homebrew/ /usr/local/bin/gem fetch fastlane --version 2.50.1
Downloaded fastlane-2.50.1

 ~/Library/Caches/Homebrew/ /usr/bin/gem fetch fastlane --version 2.50.1
Downloaded fastlane-2.50.1

But it's failing when calling with brew gem upgrade.

[nayzak]

After 2 hours of investigation I found out that problem is in Homebrew itself. It has broken something in 1.2.6 and now brew-gem has no write permissions in some places.

cd /usr/local/Homebrew/
git checkout 1.2.5
brew gem upgrade fastlane #now works fine

[nayzak]

Here is it. https://github.com/Homebrew/brew/pull/2898 https://github.com/Microsoft/homebrew-mssql-release/issues/1 Note the line from the first post ==> Using the sandbox

[nayzak]

A've made workaround for personal usage. Open /usr/local/Cellar/brew-gem/0.8.1/lib/brew/gem/cli.rb in text editor, go to line 77 and replace system "brew #{command} #{filename}" with system "brew #{command} --no-sandbox #{filename}". I'm not a Ruby developer, so I don't know the right solution how to fix this issue to make a PR.

[nicksieger]

Ah, right. And gem is trying to update the specs cache in ~/.gem/specs which is outside of the homebrew cellar and cache. Looking into whether we can get gem to skip writing the specs.

[vdupom]

Still happening to me:

$ brew gem install mailcatcher
==> Fetching mailcatcher from gem source
ERROR:  While executing gem ... (Errno::EPERM)
    Operation not permitted - /Users/luca/.gem/specs/rubygems.org%443/specs.4.8
==> /usr/bin/gem install /Users/luca/Library/Caches/Homebrew/mailcatcher-0.6.5.gem --no-ri --no-rdoc --no-wrapper --no-user-install --install-dir /usr/local/Cel
Last 15 lines from /Users/luca/Library/Logs/Homebrew/gem-mailcatcher/01.gem:
2017-09-15 12:53:16 +0200

/usr/bin/gem
install
/Users/luca/Library/Caches/Homebrew/mailcatcher-0.6.5.gem
--no-ri
--no-rdoc
--no-wrapper
--no-user-install
--install-dir
/usr/local/Cellar/gem-mailcatcher/0.6.5
--bindir
/usr/local/Cellar/gem-mailcatcher/0.6.5/bin

ERROR:  Could not find a valid gem '/Users/luca/Library/Caches/Homebrew/mailcatcher-0.6.5.gem' (>= 0) in any repository

Do not report this issue to Homebrew/brew or Homebrew/core!

My system info:

$ which ruby
/usr/local/bin/ruby
$ ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin15]
$ which gem
/usr/local/bin/gem
$ gem -v
2.6.11
$ brew --version
Homebrew 1.3.2
Homebrew/homebrew-core (git revision c79fb3; last commit 2017-09-15)
$ brew info brew-gem
brew-gem: stable 0.8.2, HEAD
Install RubyGems as Homebrew formulae
https://github.com/sportngin/brew-gem
/usr/local/Cellar/brew-gem/0.8.2 (10 files, 12.6KB) *
  Built from source on 2017-09-15 at 12:53:03
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/brew-gem.rb

[guaguanco]

I confirm that the issue is still there for me as well with brew-gem 0.8.3

[nicksieger]

@vdupom @guaguanco check that the system rubygems version is upgraded to a 2.6.x version. /usr/local/bin/ruby is only used to install brew gems if you specify the --homebrew-ruby argument. For more information about how the install is done, type brew gem formula mailcatcher to see the full homebrew formula.

[guaguanco]

Hi @nicksieger I get the same problem when I use the --homebrew-ruby argument. Same errors as @vdupom with /usr/local/bin/gem instead of /usr/bin/gem in the messages.

I rather not touch the OSX (sierra in my case) version of ruby and use homebrew's.

It seems that the gem file doesn't get downloaded in the right place because I cannot find it in the homebrew cache folder. In attachment the formula for bcat: brew-gem_bcat-formula.rb.txt.

[nicksieger]

So, this is the piece of the formula that ensures that rubygems does not try to write outside of the homebrew sandbox/cache folder:

    HOMEBREW_CACHE.cd do
      ENV['GEM_SPEC_CACHE'] = "#{HOMEBREW_CACHE}/gem_spec_cache"
      system "gem", "fetch", "bcat", "--version", resource.version
    end

Can you check if that gem_spec_cache directory was ever created? This is why I ask if somehow an older version of rubygems is used because the GEM_SPEC_CACHE environment variable is a somewhat recent addition.

[guaguanco]

The gem_spec_cache is there in the right place, but it contains just api.rubygems.org%443/latest_specs.4.8

I guess that you're right about the outdated gem, in the RubyGemsDownloadStrategy class there should be something like

rubybindir = '/usr/local/bin'
gem_path = "#{rubybindir}/gem"
system gem_path, "fetch", "bcat", "--version", resource.version

as in the GemBcat class. system "gem" is probably using the OSX outdated gem

[nicksieger]

Ah, good catch! How does #51 look?

[guaguanco]

51 looks good to me, it should fix the issue.

BTW wouldn't be better to add an explicit requirement for the gem version to brew-gem and to default to homebrew ruby if the system one is too old?