Closed flavorjones closed 6 months ago
OK, so understanding that bundler's behavior has changed in at least two unfortunate ways for the "linux + linux-musl" scenario:
Let's dig in.
On a gnu system (docker image ruby:3.2
), running ruby 3.2.3, with the following Gemfile:
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.0.1"
bundler | status |
---|---|
2.4.19 | β |
2.4.22 | β |
2.5.0 | β |
2.5.3 | β |
2.5.4 | β |
2.5.5 | β |
2.5.6 | β [1] |
Error [1]: git-bisected to rubygems/rubygems@66ee354da92fdc6a92530ecd796e1e8ab69bc1f0
Could not find gems matching 'rcee_precompiled (= 0.5.0.1)' valid for all resolution platforms (aarch64-linux-musl,
aarch64-linux, arm-linux-musl, arm-linux, arm64-darwin, x86-linux-musl, x86-linux, x86_64-darwin, x86_64-linux-musl) in
rubygems repository https://rubygems.org/ or installed locally.
The source contains the following gems matching 'rcee_precompiled (= 0.5.0.1)':
* rcee_precompiled-0.5.0.1-aarch64-linux
* rcee_precompiled-0.5.0.1-aarch64-linux-musl
* rcee_precompiled-0.5.0.1-arm-linux
* rcee_precompiled-0.5.0.1-arm-linux-musl
* rcee_precompiled-0.5.0.1-arm64-darwin
* rcee_precompiled-0.5.0.1
* rcee_precompiled-0.5.0.1-x64-mingw-ucrt
* rcee_precompiled-0.5.0.1-x64-mingw32
* rcee_precompiled-0.5.0.1-x86-linux
* rcee_precompiled-0.5.0.1-x86-linux-musl
* rcee_precompiled-0.5.0.1-x86_64-darwin
* rcee_precompiled-0.5.0.1-x86_64-linux
* rcee_precompiled-0.5.0.1-x86_64-linux-musl
On a musl system (docker image ruby:3.2-alpine
), running ruby 3.2.3, with the following Gemfile:
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.0.1"
bundler | status |
---|---|
2.4.19 | β |
2.4.20 | β |
2.4.21 | β |
2.4.22 | β |
2.5.0 | β [1] |
2.5.5 | β [1] |
2.5.6 | β [2] |
2.5.7 | β |
Error [1]: incorrectly installs x86_64-linux
instead of x86_64-linux-musl
# Gemfile.lock
GEM
remote: https://rubygems.org/
specs:
rcee_precompiled (0.5.0.1)
mini_portile2
rcee_precompiled (0.5.0.1-aarch64-linux)
rcee_precompiled (0.5.0.1-aarch64-linux-musl)
rcee_precompiled (0.5.0.1-arm-linux)
rcee_precompiled (0.5.0.1-arm-linux-musl)
rcee_precompiled (0.5.0.1-arm64-darwin)
rcee_precompiled (0.5.0.1-x86-linux)
rcee_precompiled (0.5.0.1-x86-linux-musl)
rcee_precompiled (0.5.0.1-x86_64-darwin)
rcee_precompiled (0.5.0.1-x86_64-linux)
PLATFORMS
aarch64-linux
aarch64-linux-musl
arm-linux
arm-linux-musl
arm64-darwin
ruby
x86-linux
x86-linux-musl
x86_64-darwin
x86_64-linux
DEPENDENCIES
rcee_precompiled (= 0.5.0.1)
BUNDLED WITH
2.5.0
Note [2]: https://github.com/rubygems/rubygems/issues/7432 was fixed in 2.5.6 by https://github.com/rubygems/rubygems/pull/7441
I've updated the recommendations in the description. I'll be opening a bug report on rubygems shortly to report these findings.
I've created an issue upstream at test: demonstrate issue with installing linux-musl gems by flavorjones Β· Pull Request #7583 Β· rubygems/rubygems for the issue described above in bundler 2.5.6.
Recommendations added to the README. Closing.
Debian unstable really screwed it up: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1069089
Debian unstable's ruby3.1 package currently bundles rubygems 3.4.20. However, Debian's maintainers patched it in a way that behavior like 3.3.15:
diff --git a/lib/rubygems/platform.rb b/lib/rubygems/platform.rb
index b721629b7..a16c6fca9 100644
--- a/lib/rubygems/platform.rb
+++ b/lib/rubygems/platform.rb
@@ -117,6 +117,9 @@ def initialize(arch)
when /^(\w+_platform)(\d+)?/ then [ $1, $2 ]
else [ "unknown", nil ]
end
+
+ # emulate behavior from 3.3.15 on ruby 3.1
+ @version = nil if (RUBY_VERSION < "3.2" && @os == "linux")
when Gem::Platform then
@cpu = arch.cpu
@os = arch.os
It's completely broken despite 3.4.20 > 3.3.22...
irb(main):001:0> Gem::Platform.new('aarch64-linux-gnu')
=> #<Gem::Platform:0x0000ffffa56fc8e0 @cpu="aarch64", @os="linux", @version=nil>
irb(main):002:0> Gem::Platform.new('aarch64-linux-musl')
=> #<Gem::Platform:0x0000ffffa56c8680 @cpu="aarch64", @os="linux", @version=nil>
irb(main):003:0> Gem::Platform.new('aarch64-linux-musl') == Gem::Platform.new('aarch64-linux-gnu')
=> true
I have already got a few bug reports from users on Debian unstable. What I don't want to do is to block rubygems <= 3.4.20
, because ruby3.2 on Debian unstable also have same patched rubygems 3.4.20, but works properly (see the version check in the patch above).
I have tried a few ways to reduce the noises from users by checking if they have a broken rubygems and raise a runtime error, etc. and unfortunately it brought even more noise and confusion. The final solution I end up with is to conditionally block ruby <3.2 on linux native gems, as ruby >=3.2 seems to always have a good default rubygems version (assuming users won't downgrade default rubygems).
spec.required_ruby_version = if spec.platform == Gem::Platform::RUBY || spec.platform.os != 'linux'
'>= 3.1.0'
else
'>= 3.2.0'
end
@ntkme Thanks for adding this context, that's really too bad about the downstream patches that get applied in the Sid distro. I hope it gets fixed.
We know that rubygems 3.33.22 / bundler 2.3.21 is required for basic support of
linux-gnu
andlinux-musl
platform names; but buggy bundler behavior prior to 2.5.6 may cause challenges for some users.This issue is to record some research into how the default versions behave with different native platform gems, and to document recommendations for maintainers as to how and when to ship
linux-gnu
andlinux-musl
versions.testing
To help test this, I published "linux + linux-musl" and "linux-gnu + linux-musl" combinations of the gem
rcee_precompiled
which you can find at https://rubygems.org/gems/rcee_precompiled/versions.Note that the versions in question being tested below are:
and the alpine containers have
build-base
installed to compile the source gem if necessary.We test Ruby 3.0 with updated rubygems because its default version is below the known minimum needed for this to work correctly (3.3.22).
testing
gem install
-linux
-linux
-linux
-linux-gnu
-linux-musl
-linux-musl
-linux
-linux-gnu
-linux-musl
-linux-musl
-linux
-linux-gnu
-linux-musl
-linux-musl
-linux
-linux-gnu
-linux-musl
-linux-musl
testing
bundle install
-linux
-linux-gnu
-linux-musl
-linux-musl
-linux
-linux-gnu
-linux-musl
-linux-musl
-linux
-linux-gnu
-linux-musl
-linux-musl
-linux
-linux-gnu
-linux
-linux-musl
Error [1]:
Recommendations
Given the inconsisent behavior of
gem
andbundle
when the gem in question shipslinux
andlinux-musl
platform gems, it seems pretty clear that it's not a good idea to do this.So the recommendation has to be:
-linux
platform gem that works for both gnu and musl systems,-linux-gnu
and-linux-musl
platform gems.-linux
and-linux-musl
platform gems for the same version.The installation docs for users should contain some warnings: