Closed Segaja closed 1 year ago
@segaja is there more info about your env? Is there any way to share steps to reproduce in in isolated environment like docker? I can't reproduce locally.
$ git checkout v3.3.21
$ rake test
Loaded suite /home/retro/.gem/ruby/3.1.2/gems/rake-13.0.6/lib/rake/rake_test_loader
Started
..............................................................................................................................................................................................
....................................................................................................P
==============================================================================================================================================================================================
Pending: test_operating_system_customizing_default_dir(GemTest): Temporary pending custom default_dir test
/home/retro/code/work/oss/rubygems/test/rubygems/test_rubygems.rb:48:in `rescue in test_operating_system_customizing_default_dir'
/home/retro/code/work/oss/rubygems/test/rubygems/test_rubygems.rb:45:in `test_operating_system_customizing_default_dir'
42: "require \"rubygems\"; puts Gem::Specification.stubs.map(&:full_name)",
43: { :err => [:child, :out] }
44: ).strip
=> 45: begin
46: assert_empty output
47: rescue Test::Unit::AssertionFailedError
48: pend "Temporary pending custom default_dir test"
==============================================================================================================================================================================================
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
...................................................................................................................
Finished in 153.651654711 seconds.
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2306 tests, 6358 assertions, 0 failures, 0 errors, 1 pendings, 0 omissions, 0 notifications
99.9566% passed
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
15.01 tests/s, 41.38 assertions/s
it is a clean chroot in a ruby 3.0.4 setup. (sorry i just realize that is not "latest" :)
same with 3.0.4
Loaded suite /home/retro/.gem/ruby/3.0.4/gems/rake-13.0.6/lib/rake/rake_test_loader
Started
..............................................................................................................................................................................................
....................................................................................................P
==============================================================================================================================================================================================
Pending: test_operating_system_customizing_default_dir(GemTest): Temporary pending custom default_dir test
/home/retro/code/work/oss/rubygems/test/rubygems/test_rubygems.rb:48:in `rescue in test_operating_system_customizing_default_dir'
/home/retro/code/work/oss/rubygems/test/rubygems/test_rubygems.rb:45:in `test_operating_system_customizing_default_dir'
42: "require \"rubygems\"; puts Gem::Specification.stubs.map(&:full_name)",
43: { :err => [:child, :out] }
44: ).strip
=> 45: begin
46: assert_empty output
47: rescue Test::Unit::AssertionFailedError
48: pend "Temporary pending custom default_dir test"
==============================================================================================================================================================================================
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
..............................................................................................................................................................................................
...................................................................................................................
Finished in 142.566131683 seconds.
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2306 tests, 6358 assertions, 0 failures, 0 errors, 1 pendings, 0 omissions, 0 notifications
99.9566% passed
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
16.17 tests/s, 44.60 assertions/s
Is there any command I can run in the chroot before the tests to give you more informations?
@Segaja not sure. You'll need to check why the test is failing on your own or share steps to reproduce in some isolated environment like docker to make it easy to reproduce for me (or other maintainers).
@simi maybe this information helps:
+ ruby --version
ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux]
+ gem env
RubyGems Environment:
- RUBYGEMS VERSION: 3.3.21
- RUBY VERSION: 3.0.4 (2022-04-12 patchlevel 208) [x86_64-linux]
- INSTALLATION DIRECTORY: /usr/lib/ruby/gems/3.0.0
- USER INSTALLATION DIRECTORY: /build/.local/share/gem/ruby/3.0.0
- RUBY EXECUTABLE: /usr/bin/ruby
- GIT EXECUTABLE: /usr/bin/git
- EXECUTABLE DIRECTORY: /usr/bin
- SPEC CACHE DIRECTORY: /build/.local/share/gem/specs
- SYSTEM CONFIGURATION DIRECTORY: /etc
- RUBYGEMS PLATFORMS:
- ruby
- x86_64-linux
- GEM PATHS:
- /usr/lib/ruby/gems/3.0.0
- /build/.local/share/gem/ruby/3.0.0
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :backtrace => true
- :bulk_threshold => 1000
- "gem" => "--user-install"
- REMOTE SOURCES:
- https://rubygems.org/
- SHELL PATH:
- /build/.local/share/gem/ruby/3.0.0/bin
- /usr/local/sbin
- /usr/local/bin
- /usr/bin
- /usr/bin/site_perl
- /usr/bin/vendor_perl
- /usr/bin/core_perl
+ gem list --local
abbrev (default: 0.1.0)
base64 (default: 0.1.1)
benchmark (default: 0.2.0)
bigdecimal (default: 3.1.2)
bundler (default: 2.3.22)
cgi (default: 0.3.2)
csv (default: 3.2.5)
date (default: 3.2.2)
dbm (default: 1.1.0)
debug (default: 0.2.1)
delegate (default: 0.2.0)
did_you_mean (default: 1.6.1)
digest (default: 3.1.0)
drb (default: 2.1.0)
english (default: 0.7.1)
erb (default: 2.2.3)
etc (default: 1.3.0)
fcntl (default: 1.0.1)
fiddle (default: 1.1.0)
fileutils (default: 1.6.0)
find (default: 0.1.1)
forwardable (default: 1.3.2)
gdbm (default: 2.1.0)
getoptlong (default: 0.1.1)
io-console (default: 0.5.11)
io-nonblock (default: 0.1.0)
io-wait (default: 0.2.3)
ipaddr (default: 1.2.4)
irb (default: 1.4.1)
json (default: 2.6.2)
logger (default: 1.5.1)
matrix (default: 0.3.1)
minitest (5.16.3)
mutex_m (default: 0.1.1)
net-ftp (default: 0.1.2)
net-http (default: 0.2.2)
net-imap (default: 0.1.1)
net-pop (default: 0.1.1)
net-protocol (default: 0.1.1)
net-smtp (default: 0.2.1)
nkf (default: 0.1.0)
observer (default: 0.1.1)
open-uri (default: 0.2.0)
open3 (default: 0.1.1)
openssl (default: 2.2.1)
optparse (default: 0.1.1)
ostruct (default: 0.3.1)
pathname (default: 0.1.0)
power_assert (2.0.1)
pp (default: 0.2.1)
prettyprint (default: 0.1.1)
prime (default: 0.1.2)
pstore (default: 0.1.1)
psych (default: 4.0.4)
racc (default: 1.6.0)
rake (13.0.6)
rdoc (default: 6.4.0)
readline (default: 0.0.2)
readline-ext (default: 0.1.1)
reline (default: 0.3.1)
resolv (default: 0.2.1)
resolv-replace (default: 0.1.0)
rexml (3.2.5)
rinda (default: 0.1.1)
ruby2_keywords (0.0.5)
securerandom (default: 0.1.0)
set (default: 1.0.1)
shellwords (default: 0.1.0)
singleton (default: 0.1.1)
stringio (default: 3.0.2)
strscan (default: 3.0.1)
syslog (default: 0.1.0)
tempfile (default: 0.1.1)
test-unit (3.5.3)
time (default: 0.2.0)
timeout (default: 0.1.1)
tmpdir (default: 0.1.2)
tracer (default: 0.1.1)
tsort (default: 0.1.0)
un (default: 0.1.0)
uri (default: 0.11.0)
weakref (default: 0.1.1)
webrick (1.7.0)
yaml (default: 0.1.1)
zlib (default: 2.0.0)
The only unusual thingie I see is "gem" => "--user-install"
. Can you try to run tests without this config? Maybe we rely on some default configuration not present thanks to this option.
i found a way to reproduce it in docker. Here is the Dockerfile
:
FROM archlinux
RUN echo -e "[community-testing]\nInclude = /etc/pacman.d/mirrorlist" >> /etc/pacman.conf \
&& useradd --create-home build \
&& pacman --sync --refresh --sysupgrade --noconfirm --needed \
# cargo \
community-testing/ruby \
community-testing/ruby-cgi \
community-testing/ruby-io-nonblock \
community-testing/ruby-json \
# cmake \
devtools \
gcc \
git \
make \
ruby-rake \
ruby-webrick
USER build
WORKDIR /home/build
RUN git clone https://github.com/rubygems/rubygems.git \
&& cd rubygems \
&& git checkout "v3.3.21"
WORKDIR /home/build/rubygems
RUN rake test --trace
I solved the last error about the "Run rake update_manifest
to sync it" error.
@simi any more ideas about this?
The only unusual thingie I see is
"gem" => "--user-install"
. Can you try to run tests without this config? Maybe we rely on some default configuration not present thanks to this option.
I tried it without that setting but had no effect on the result.
I also noticed that both tests are working with the json
gem. Maybe somehow that is the problem. In my environment json is in 2.6.2 and not 2.5.1. Can this cause issues?
docker build .
Sending build context to Docker daemon 2.56kB
Step 1/7 : FROM archlinux
---> 7b17a3e650d6
Step 2/7 : RUN echo -e "[community-testing]\nInclude = /etc/pacman.d/mirrorlist" >> /etc/pacman.conf && useradd --create-home build && pacman --sync --refresh --sysupgrade --noconfirm --needed community-testing/ruby community-testing/ruby-cgi community-testing/ruby-io-nonblock community-testing/ruby-json devtools gcc git make ruby-rake ruby-webrick
---> Running in e561ca19260e
:: Synchronizing package databases...
core downloading...
extra downloading...
community downloading...
community-testing downloading...
error: target not found: ruby
error: target not found: ruby-cgi
error: target not found: ruby-io-nonblock
error: target not found: ruby-json
The command '/bin/sh -c echo -e "[community-testing]\nInclude = /etc/pacman.d/mirrorlist" >> /etc/pacman.conf && useradd --create-home build && pacman --sync --refresh --sysupgrade --noconfirm --needed community-testing/ruby community-testing/ruby-cgi community-testing/ruby-io-nonblock community-testing/ruby-json devtools gcc git make ruby-rake ruby-webrick' returned a non-zero code: 1
Is there anything I do wrong?
Sorry. Small changes:
FROM archlinux
RUN echo -e "[community-testing]\nInclude = /etc/pacman.d/mirrorlist" > /etc/pacman.conf.new \
&& cat /etc/pacman.conf >> /etc/pacman.conf.new \
&& mv /etc/pacman.conf.new /etc/pacman.conf \
&& useradd --create-home build \
&& pacman --sync --refresh --sysupgrade --noconfirm --needed \
# cargo \
ruby \
ruby-cgi \
ruby-io-nonblock \
ruby-json \
# cmake \
devtools \
gcc \
git \
make \
ruby-rake \
ruby-webrick \
&& rm -f /etc/gemrc
USER build
WORKDIR /home/build
RUN git clone https://github.com/rubygems/rubygems.git \
&& cd rubygems \
&& git checkout "v3.3.23"
WORKDIR /home/build/rubygems
RUN rake test --trace
It seems the problem is somewhere in the ruby packaged by archlinux.
This is expected to return specification.
ruby -rjson -e "p Gem.loaded_specs['json']"
But on archlinux's Ruby (in your Dockerfile) it returns nil.
Yeah I also noticed that one, but so far I can't find any issue in the json gem in ruby. Do you happen to have any idea what could be wrong?
That seems not related to Ruby or RubyGems or JSON gem itself. It is something in Ruby package of archlinux. Can you check you can reproduce this by installing original Ruby (using ruby-install for example) and try the code above on archlinux?
so 8. 10. 2022 v 19:49 odesílatel Andreas Schleifer < @.***> napsal:
Yeah I also noticed that one, but so far I can't find any issue in the json gem in ruby. Do you happen to have any idea what could be wrong?
— Reply to this email directly, view it on GitHub https://github.com/rubygems/rubygems/issues/5951#issuecomment-1272367211, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABPLEAZPUKTFC6OP33R5LTWCGXZRANCNFSM6AAAAAAQVFLS4I . You are receiving this because you were mentioned.Message ID: @.***>
In addition I suggest to check the contents of the defaults/operating_system.rb
file in the Ruby (or RubyGems) archlinux pacakge. I think that's most likely the culprit.
In addition I suggest to check the contents of the
defaults/operating_system.rb
file in the Ruby (or RubyGems) archlinux pacakge. I think that's most likely the culprit.
This file doesn't exist on archlinux and we also don't remove it from the ruby package.
It seems it is removing stdlib and packaging it separatelly. If I understand it well, this is not officially supported.
https://github.com/archlinux/svntogit-community/blob/packages/ruby/trunk/PKGBUILD https://github.com/archlinux/svntogit-community/blob/packages/ruby-json/trunk/PKGBUILD
Yes we ship them separately. So far we had no problem with it. This is the first time it doesn't work and I don't understand why.
running this with a ruby installation from ruby-install
i get different errors but not the json related errors.
feel free to share your errors
** Invoke test (first_time)
** Execute test
Loaded suite /usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/rake_test_loader
Started
...............................................................................................................................................................
...............................................................................................................................................................
...............................................................................................................................................................
...............................................................................................................................................................
...............................................................................................................................................................
...............................................................................................................................................................
................................F
===============================================================================================================================================================
87: assert_match d, d, "match self"
88: assert_match dep("a", ">= 0"), d, "match version exact"
89: assert_match dep("a", ">= 0"), dep("a", "1"), "match version"
=> 90: refute_match dep("a"), Object.new
91:
92: Gem::Deprecate.skip_during do
93: assert_match dep(/a/, ">= 0"), d, "match simple regexp"
/home/build/rubygems/test/rubygems/test_gem_dependency.rb:90:in `test_equals_tilde'
Failure: test_equals_tilde(TestGemDependency):
<REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
<Gem::Dependency.new("a", Gem::Requirement.new([">= 0"]), :runtime)> was expected to be instance_of?
<Regexp> but was
<Gem::Dependency>.
===============================================================================================================================================================
F
===============================================================================================================================================================
97: end
98:
99: def test_equals_tilde_escape
=> 100: refute_match dep("a|b"), dep("a", "1")
101: Gem::Deprecate.skip_during do
102: assert_match dep(/a|b/), dep("a", "1")
103: end
/home/build/rubygems/test/rubygems/test_gem_dependency.rb:100:in `test_equals_tilde_escape'
Failure: test_equals_tilde_escape(TestGemDependency):
<REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
<Gem::Dependency.new("a|b", Gem::Requirement.new([">= 0"]), :runtime)> was expected to be instance_of?
<Regexp> but was
<Gem::Dependency>.
===============================================================================================================================================================
.F
===============================================================================================================================================================
/home/build/rubygems/test/rubygems/test_gem_dependency.rb:117:in `test_equals_tilde_spec'
/home/build/rubygems/lib/rubygems/deprecate.rb:86:in `skip_during'
117: Gem::Deprecate.skip_during do
118: assert_match dep(/a/, ">= 0"), spec("a", "0")
119: assert_match dep(/a|b/, ">= 0"), spec("b", "0")
=> 120: refute_match dep(/a/, ">= 0"), spec("b", "0")
121: end
122: end
123:
/home/build/rubygems/test/rubygems/test_gem_dependency.rb:120:in `block in test_equals_tilde_spec'
Failure: test_equals_tilde_spec(TestGemDependency):
<REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
<Gem::Dependency.new(/a/, Gem::Requirement.new([">= 0"]), :runtime)> was expected to be instance_of?
<Regexp> but was
<Gem::Dependency>.
===============================================================================================================================================================
....................................................................................................................P
===============================================================================================================================================================
42: end
43:
44: def test_build_cdylib
=> 45: skip_unsupported_platforms!
46: setup_rust_gem "rust_ruby_example"
47:
48: output = []
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:45:in `test_build_cdylib'
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:169:in `skip_unsupported_platforms!'
Pending: test_build_cdylib(TestGemExtCargoBuilder): cargo not present
===============================================================================================================================================================
P
===============================================================================================================================================================
66: end
67:
68: def test_build_dev_profile
=> 69: skip_unsupported_platforms!
70: setup_rust_gem "rust_ruby_example"
71:
72: output = []
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:69:in `test_build_dev_profile'
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:169:in `skip_unsupported_platforms!'
Pending: test_build_dev_profile(TestGemExtCargoBuilder): cargo not present
===============================================================================================================================================================
P
===============================================================================================================================================================
91: end
92:
93: def test_build_fail
=> 94: skip_unsupported_platforms!
95: setup_rust_gem "rust_ruby_example"
96:
97: output = []
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:94:in `test_build_fail'
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:169:in `skip_unsupported_platforms!'
Pending: test_build_fail(TestGemExtCargoBuilder): cargo not present
===============================================================================================================================================================
P
===============================================================================================================================================================
23: end
24:
25: def test_build_staticlib
=> 26: skip_unsupported_platforms!
27: setup_rust_gem "rust_ruby_example"
28:
29: content = @fixture_dir.join("Cargo.toml").read.gsub("cdylib", "staticlib")
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:26:in `test_build_staticlib'
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:169:in `skip_unsupported_platforms!'
Pending: test_build_staticlib(TestGemExtCargoBuilder): cargo not present
===============================================================================================================================================================
P
===============================================================================================================================================================
137: end
138:
139: def test_custom_name
=> 140: skip_unsupported_platforms!
141: setup_rust_gem "custom_name"
142:
143: Dir.chdir @ext do
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:140:in `test_custom_name'
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:169:in `skip_unsupported_platforms!'
Pending: test_custom_name(TestGemExtCargoBuilder): cargo not present
===============================================================================================================================================================
P
===============================================================================================================================================================
113: end
114:
115: def test_full_integration
=> 116: skip_unsupported_platforms!
117: setup_rust_gem "rust_ruby_example"
118:
119: require "open3"
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:116:in `test_full_integration'
/home/build/rubygems/test/rubygems/test_gem_ext_cargo_builder.rb:169:in `skip_unsupported_platforms!'
Pending: test_full_integration(TestGemExtCargoBuilder): cargo not present
===============================================================================================================================================================
......................P
===============================================================================================================================================================
/home/build/rubygems/test/rubygems/test_gem_ext_cmake_builder.rb:14:in `setup'
/home/build/rubygems/test/rubygems/test_gem_ext_cmake_builder.rb:18:in `rescue in setup'
Pending: test_self_build(TestGemExtCmakeBuilder): cmake not present
===============================================================================================================================================================
P
===============================================================================================================================================================
/home/build/rubygems/test/rubygems/test_gem_ext_cmake_builder.rb:14:in `setup'
/home/build/rubygems/test/rubygems/test_gem_ext_cmake_builder.rb:18:in `rescue in setup'
Pending: test_self_build_fail(TestGemExtCmakeBuilder): cmake not present
===============================================================================================================================================================
P
===============================================================================================================================================================
/home/build/rubygems/test/rubygems/test_gem_ext_cmake_builder.rb:14:in `setup'
/home/build/rubygems/test/rubygems/test_gem_ext_cmake_builder.rb:18:in `rescue in setup'
Pending: test_self_build_has_makefile(TestGemExtCmakeBuilder): cmake not present
===============================================================================================================================================================
..............................................................................................F
===============================================================================================================================================================
419: # oddballs
420: assert_local_match "i386-mswin32-mq5.3"
421: assert_local_match "i386-mswin32-mq6"
=> 422: refute_local_match "win32-1.8.2-VC7"
423: refute_local_match "win32-1.8.4-VC6"
424: refute_local_match "win32-source"
425: refute_local_match "windows"
/home/build/rubygems/test/rubygems/test_gem_platform.rb:422:in `test_equals_tilde'
/home/build/rubygems/test/rubygems/test_gem_platform.rb:495:in `refute_local_match'
Failure: test_equals_tilde(TestGemPlatform):
<REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
<#<Gem::Platform:0x00000cef313c5528 @cpu="x86", @os="mswin32", @version="60">> was expected to be instance_of?
<Regexp> but was
<Gem::Platform>.
===============================================================================================================================================================
...............................................................................................................................................................
...............................................................................................................................................................
...............................................................................................................................................................
...............................................................................................................................................F
===============================================================================================================================================================
21:
22: sf = Gem::SourceFetchProblem.new source, error
23:
=> 24: refute_match sf.wordy, "secret"
25: end
26:
27: def test_source_password_no_redacted
/home/build/rubygems/test/rubygems/test_gem_source_fetch_problem.rb:24:in `test_password_redacted'
Failure: test_password_redacted(TestGemSourceFetchProblem):
<REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
<"Unable to download data from https://username:REDACTED@gemsource.com - test"> was expected to be instance_of?
<Regexp> but was
<String>.
===============================================================================================================================================================
...............................................................................................................................................................
...............................................................................................................................................................
..................................................................................................................................................
Finished in 26.580760107 seconds.
---------------------------------------------------------------------------------------------------------------------------------------------------------------
2317 tests, 6343 assertions, 5 failures, 0 errors, 9 pendings, 0 omissions, 0 notifications
99.3958% passed
---------------------------------------------------------------------------------------------------------------------------------------------------------------
87.17 tests/s, 238.63 assertions/s
rake aborted!
Command failed with status (1): [ruby -w -I"lib:test:bundler/lib" -w /usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/rake_test_loader.rb "test/rubygems/test_bundled_ca.rb" "test/rubygems/test_config.rb" "test/rubygems/test_deprecate.rb" "test/rubygems/test_exit.rb" "test/rubygems/test_gem.rb" "test/rubygems/test_gem_available_set.rb" "test/rubygems/test_gem_bundler_version_finder.rb" "test/rubygems/test_gem_command.rb" "test/rubygems/test_gem_command_manager.rb" "test/rubygems/test_gem_commands_build_command.rb" "test/rubygems/test_gem_commands_cert_command.rb" "test/rubygems/test_gem_commands_check_command.rb" "test/rubygems/test_gem_commands_cleanup_command.rb" "test/rubygems/test_gem_commands_contents_command.rb" "test/rubygems/test_gem_commands_dependency_command.rb" "test/rubygems/test_gem_commands_environment_command.rb" "test/rubygems/test_gem_commands_fetch_command.rb" "test/rubygems/test_gem_commands_generate_index_command.rb" "test/rubygems/test_gem_commands_help_command.rb" "test/rubygems/test_gem_commands_info_command.rb" "test/rubygems/test_gem_commands_install_command.rb" "test/rubygems/test_gem_commands_list_command.rb" "test/rubygems/test_gem_commands_lock_command.rb" "test/rubygems/test_gem_commands_mirror.rb" "test/rubygems/test_gem_commands_open_command.rb" "test/rubygems/test_gem_commands_outdated_command.rb" "test/rubygems/test_gem_commands_owner_command.rb" "test/rubygems/test_gem_commands_pristine_command.rb" "test/rubygems/test_gem_commands_push_command.rb" "test/rubygems/test_gem_commands_query_command.rb" "test/rubygems/test_gem_commands_search_command.rb" "test/rubygems/test_gem_commands_server_command.rb" "test/rubygems/test_gem_commands_setup_command.rb" "test/rubygems/test_gem_commands_signin_command.rb" "test/rubygems/test_gem_commands_signout_command.rb" "test/rubygems/test_gem_commands_sources_command.rb" "test/rubygems/test_gem_commands_specification_command.rb" "test/rubygems/test_gem_commands_stale_command.rb" "test/rubygems/test_gem_commands_uninstall_command.rb" "test/rubygems/test_gem_commands_unpack_command.rb" "test/rubygems/test_gem_commands_update_command.rb" "test/rubygems/test_gem_commands_which_command.rb" "test/rubygems/test_gem_commands_yank_command.rb" "test/rubygems/test_gem_config_file.rb" "test/rubygems/test_gem_dependency.rb" "test/rubygems/test_gem_dependency_installer.rb" "test/rubygems/test_gem_dependency_list.rb" "test/rubygems/test_gem_dependency_resolution_error.rb" "test/rubygems/test_gem_doctor.rb" "test/rubygems/test_gem_ext_builder.rb" "test/rubygems/test_gem_ext_cargo_builder.rb" "test/rubygems/test_gem_ext_cargo_builder_link_flag_converter.rb" "test/rubygems/test_gem_ext_cargo_builder_unit.rb" "test/rubygems/test_gem_ext_cmake_builder.rb" "test/rubygems/test_gem_ext_configure_builder.rb" "test/rubygems/test_gem_ext_ext_conf_builder.rb" "test/rubygems/test_gem_ext_rake_builder.rb" "test/rubygems/test_gem_gem_runner.rb" "test/rubygems/test_gem_gemcutter_utilities.rb" "test/rubygems/test_gem_impossible_dependencies_error.rb" "test/rubygems/test_gem_indexer.rb" "test/rubygems/test_gem_install_update_options.rb" "test/rubygems/test_gem_installer.rb" "test/rubygems/test_gem_local_remote_options.rb" "test/rubygems/test_gem_name_tuple.rb" "test/rubygems/test_gem_package.rb" "test/rubygems/test_gem_package_old.rb" "test/rubygems/test_gem_package_tar_header.rb" "test/rubygems/test_gem_package_tar_reader.rb" "test/rubygems/test_gem_package_tar_reader_entry.rb" "test/rubygems/test_gem_package_tar_writer.rb" "test/rubygems/test_gem_package_task.rb" "test/rubygems/test_gem_path_support.rb" "test/rubygems/test_gem_platform.rb" "test/rubygems/test_gem_rdoc.rb" "test/rubygems/test_gem_remote_fetcher.rb" "test/rubygems/test_gem_request.rb" "test/rubygems/test_gem_request_connection_pools.rb" "test/rubygems/test_gem_request_set.rb" "test/rubygems/test_gem_request_set_gem_dependency_api.rb" "test/rubygems/test_gem_request_set_lockfile.rb" "test/rubygems/test_gem_request_set_lockfile_parser.rb" "test/rubygems/test_gem_request_set_lockfile_tokenizer.rb" "test/rubygems/test_gem_requirement.rb" "test/rubygems/test_gem_resolver.rb" "test/rubygems/test_gem_resolver_activation_request.rb" "test/rubygems/test_gem_resolver_api_set.rb" "test/rubygems/test_gem_resolver_api_specification.rb" "test/rubygems/test_gem_resolver_best_set.rb" "test/rubygems/test_gem_resolver_composed_set.rb" "test/rubygems/test_gem_resolver_conflict.rb" "test/rubygems/test_gem_resolver_dependency_request.rb" "test/rubygems/test_gem_resolver_git_set.rb" "test/rubygems/test_gem_resolver_git_specification.rb" "test/rubygems/test_gem_resolver_index_set.rb" "test/rubygems/test_gem_resolver_index_specification.rb" "test/rubygems/test_gem_resolver_installed_specification.rb" "test/rubygems/test_gem_resolver_installer_set.rb" "test/rubygems/test_gem_resolver_local_specification.rb" "test/rubygems/test_gem_resolver_lock_set.rb" "test/rubygems/test_gem_resolver_lock_specification.rb" "test/rubygems/test_gem_resolver_requirement_list.rb" "test/rubygems/test_gem_resolver_specification.rb" "test/rubygems/test_gem_resolver_vendor_set.rb" "test/rubygems/test_gem_resolver_vendor_specification.rb" "test/rubygems/test_gem_security.rb" "test/rubygems/test_gem_security_policy.rb" "test/rubygems/test_gem_security_signer.rb" "test/rubygems/test_gem_security_trust_dir.rb" "test/rubygems/test_gem_silent_ui.rb" "test/rubygems/test_gem_source.rb" "test/rubygems/test_gem_source_fetch_problem.rb" "test/rubygems/test_gem_source_git.rb" "test/rubygems/test_gem_source_installed.rb" "test/rubygems/test_gem_source_list.rb" "test/rubygems/test_gem_source_local.rb" "test/rubygems/test_gem_source_lock.rb" "test/rubygems/test_gem_source_specific_file.rb" "test/rubygems/test_gem_source_subpath_problem.rb" "test/rubygems/test_gem_source_vendor.rb" "test/rubygems/test_gem_spec_fetcher.rb" "test/rubygems/test_gem_specification.rb" "test/rubygems/test_gem_stream_ui.rb" "test/rubygems/test_gem_stub_specification.rb" "test/rubygems/test_gem_text.rb" "test/rubygems/test_gem_uninstaller.rb" "test/rubygems/test_gem_unsatisfiable_dependency_error.rb" "test/rubygems/test_gem_uri.rb" "test/rubygems/test_gem_uri_formatter.rb" "test/rubygems/test_gem_util.rb" "test/rubygems/test_gem_validator.rb" "test/rubygems/test_gem_version.rb" "test/rubygems/test_gem_version_option.rb" "test/rubygems/test_kernel.rb" "test/rubygems/test_project_sanity.rb" "test/rubygems/test_remote_fetch_error.rb" "test/rubygems/test_require.rb" "test/rubygems/test_rubygems.rb" "test/test_changelog_generator.rb" ]
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/testtask.rb:130:in `block (3 levels) in define'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/file_utils.rb:57:in `sh'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/file_utils.rb:104:in `ruby'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/testtask.rb:117:in `block (2 levels) in define'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/file_utils_ext.rb:58:in `verbose'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/testtask.rb:111:in `block in define'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/task.rb:281:in `block in execute'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/task.rb:281:in `each'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/task.rb:281:in `execute'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/task.rb:199:in `synchronize'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/task.rb:199:in `invoke_with_call_chain'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/task.rb:188:in `invoke'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:160:in `invoke_task'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:116:in `each'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:116:in `block in top_level'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:125:in `run_with_threads'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:110:in `top_level'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:83:in `block in run'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:186:in `standard_exception_handling'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/lib/rake/application.rb:80:in `run'
/usr/local/lib/ruby/gems/3.0.0/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
/usr/local/bin/rake:25:in `load'
/usr/local/bin/rake:25:in `<main>'
Tasks: TOP => test
But I guess this doesn't explain what happened to the json
gem.
My guess is that json
is not a default gem in archlinux, so those realworld tests can´t possibly pass since they rely on json being a default gem (and thus get automatically upgraded if a higher version of itself is also installed).
That seems a defect of the packaging process to me. Even if ruby-json
is shipped as a separate package, the ruby
package should still come with its own copy installed as a default gem that gets activated when `json is required. I guess you could look at how other distributions handle this, but in any case, this does not seem like an upstream issue.
Regarding the last failures copied, maybe upgrading test-unit
will fix those.
Well Arch is one of the first distributions who does this. And ruby-json
copies the files into the same structure as the default ruby
package has (or in arch case had) them.
I'm going to diff the json changes from ruby-json
with the ones from ruby upstream.
What does gem list json
print? It should say
$ gem list json
*** LOCAL GEMS ***
json (default: 2.5.1)
as per https://stdgems.org/json/.
╰─ gem list json
*** LOCAL GEMS ***
json (default: 2.6.2)
So that's not it :thinking:.
Something else you may want to check is whether the json
gem is being properly registered here: https://github.com/rubygems/rubygems/blob/629e08d34d14f60fe3398e40fc72f4f171ffc582/lib/rubygems.rb#L1191-L1211.
In order for json
to be properly activated as a gem when required, that @path_to_default_spec_map
should have a key named ¨json".
how do i see what is in this field? just puts
it?
Yeah you can puts
it!
╰─ irb
irb(main):001:0> puts @path_to_default_spec_map
=> nil
no output from puts @path_to_default_spec_map
in ruby -e
.
It's an instance variable in the Gem
module. Try Gem.instance_variable_get(:@path_to_default_spec_map)["json"]
in ruby -e
i see no output.
In irb I see an output for puts Gem.instance_variable_get(:@path_to_default_spec_map)
but not for puts Gem.instance_variable_get(:@path_to_default_spec_map)["json"]
Well that would confirm that while other default gems are being properly registered as default gems, the json gem is not. The code that uses that method to register default gems is here:
So probably a json
gemspec is missing at Gem.default_specifications_dir
?
╰─ ruby -e 'puts Gem.default_specifications_dir'
/usr/lib/ruby/gems/3.0.0/specifications/default
╰─ ls -lah /usr/lib/ruby/gems/3.0.0/specifications/default/json-2.6.2.gemspec
-rw-r--r-- 1 root root 1.7K Sep 21 22:17 /usr/lib/ruby/gems/3.0.0/specifications/default/json-2.6.2.gemspec
This is the content of the file:
# -*- encoding: utf-8 -*-
# stub: json 2.6.2 ruby lib
# stub: ext/json/ext/generator/extconf.rbext/json/ext/parser/extconf.rbext/json/extconf.rb
Gem::Specification.new do |s|
s.name = "json".freeze
s.version = "2.6.2"
s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
s.metadata = { "bug_tracker_uri" => "https://github.com/flori/json/issues", "changelog_uri" => "https://github.com/flori/json/blob/master/CHANGES.md", "documentation_uri" => "http://flori.github.io/json/doc/index.html", "homepage_uri" => "http://flori.github.io/json/", "source_code_uri" => "https://github.com/flori/json", "wiki_uri" => "https://github.com/flori/json/wiki" } if s.respond_to? :metadata=
s.require_paths = ["lib".freeze]
s.authors = ["Florian Frank".freeze]
s.date = "2022-10-08"
s.description = "This is a JSON implementation as a Ruby extension in C.".freeze
s.email = "flori@ping.de".freeze
s.extensions = ["ext/json/ext/generator/extconf.rb".freeze, "ext/json/ext/parser/extconf.rb".freeze, "ext/json/extconf.rb".freeze]
s.extra_rdoc_files = ["README.md".freeze]
s.files = ["README.md".freeze, "ext/json/ext/generator/extconf.rb".freeze, "ext/json/ext/parser/extconf.rb".freeze, "ext/json/extconf.rb".freeze]
s.homepage = "http://flori.github.com/json".freeze
s.licenses = ["Ruby".freeze]
s.rdoc_options = ["--title".freeze, "JSON implementation for Ruby".freeze, "--main".freeze, "README.md".freeze]
s.required_ruby_version = Gem::Requirement.new(">= 2.3".freeze)
s.rubygems_version = "3.3.21".freeze
s.summary = "JSON Implementation for Ruby".freeze
s.installed_by_version = "3.3.21" if s.respond_to? :installed_by_version
end
strange enough sometimes it registeres as binary and the second # stub: ...
line has some binary chars in it. but even if i remove the stub lines it doesn't work.
Oh, that's the problem right there, there's no "json.rb" assigned to files=
, only extension files and README in there. See what the file in my ruby installation looks like:
# -*- encoding: utf-8 -*-
# stub: json 2.6.1 ruby lib
# stub: ext/json/ext/generator/extconf.rbext/json/ext/parser/extconf.rbext/json/extconf.rb
Gem::Specification.new do |s|
s.name = "json".freeze
s.version = "2.6.1"
s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
s.metadata = { "bug_tracker_uri" => "https://github.com/flori/json/issues", "changelog_uri" => "https://github.com/flori/json/blob/master/CHANGES.md", "documentation_uri" => "http://flori.github.io/json/doc/index.html", "homepage_uri" => "http://flori.github.io/json/", "source_code_uri" => "https://github.com/flori/json", "wiki_uri" => "https://github.com/flori/json/wiki" } if s.respond_to? :metadata=
s.require_paths = ["lib".freeze]
s.authors = ["Florian Frank".freeze]
s.date = "2022-10-04"
s.description = "This is a JSON implementation as a Ruby extension in C.".freeze
s.email = "flori@ping.de".freeze
s.extensions = ["ext/json/ext/generator/extconf.rb".freeze, "ext/json/ext/parser/extconf.rb".freeze, "ext/json/extconf.rb".freeze]
s.extra_rdoc_files = ["README.md".freeze]
s.files = ["README.md".freeze, "ext/json/ext/generator/extconf.rb".freeze, "ext/json/ext/parser/extconf.rb".freeze, "ext/json/extconf.rb".freeze, "json.rb".freeze, "json/add/bigdecimal.rb".freeze, "json/add/complex.rb".freeze, "json/add/core.rb".freeze, "json/add/date.rb".freeze, "json/add/date_time.rb".freeze, "json/add/exception.rb".freeze, "json/add/ostruct.rb".freeze, "json/add/range.rb".freeze, "json/add/rational.rb".freeze, "json/add/regexp.rb".freeze, "json/add/set.rb".freeze, "json/add/struct.rb".freeze, "json/add/symbol.rb".freeze, "json/add/time.rb".freeze, "json/common.rb".freeze, "json/ext.rb".freeze, "json/ext/generator.so".freeze, "json/ext/parser.so".freeze, "json/generic_object.rb".freeze, "json/version.rb".freeze]
s.homepage = "http://flori.github.com/json".freeze
s.licenses = ["Ruby".freeze]
s.rdoc_options = ["--title".freeze, "JSON implementation for Ruby".freeze, "--main".freeze, "README.md".freeze]
s.required_ruby_version = Gem::Requirement.new(">= 2.3".freeze)
s.rubygems_version = "3.3.7".freeze
s.summary = "JSON Implementation for Ruby".freeze
end
or see the source file here: https://github.com/flori/json/blob/572c9994648fca172a19026f9a665e92500864ba/json.gemspec.
Hm I don't get this... we use rake compile build
( https://github.com/archlinux/svntogit-community/blob/packages/ruby-json/trunk/PKGBUILD#L29-L30 ) and the files list gets reduced during that build. the same result when i use gem build json.gemspec
.
You'll have to investigate what you're doing wrong when building the gem. The gem built at https://rubygems.org/gems/json has the proper file list.
I'm going to close this since this is not a RubyGems issue, but I'm still happy to help.
But even if I gem install
the downloaded version from https://rubygems.org/gems/json the resulting json-2.6.2.gemspec
does not contain all these files:
# -*- encoding: utf-8 -*-
# stub: json 2.6.2 ruby lib
# stub: ext/json/ext/generator/extconf.rbext/json/ext/parser/extconf.rbext/json/extconf.rb
Gem::Specification.new do |s|
s.name = "json".freeze
s.version = "2.6.2"
s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
s.metadata = { "bug_tracker_uri" => "https://github.com/flori/json/issues", "changelog_uri" => "https://github.com/flori/json/blob/master/CHANGES.md", "documentation_uri" => "http://flori.github.io/json/doc/index.html", "homepage_uri" => "http://flori.github.io/json/", "source_code_uri" => "https://github.com/flori/json", "wiki_uri" => "https://github.com/flori/json/wiki" } if s.respond_to? :metadata=
s.require_paths = ["lib".freeze]
s.authors = ["Florian Frank".freeze]
s.date = "2022-05-16"
s.description = "This is a JSON implementation as a Ruby extension in C.".freeze
s.email = "flori@ping.de".freeze
s.extensions = ["ext/json/ext/generator/extconf.rb".freeze, "ext/json/ext/parser/extconf.rb".freeze, "ext/json/extconf.rb".freeze]
s.extra_rdoc_files = ["README.md".freeze]
s.files = ["README.md".freeze, "ext/json/ext/generator/extconf.rb".freeze, "ext/json/ext/parser/extconf.rb".freeze, "ext/json/extconf.rb".freeze]
s.homepage = "http://flori.github.com/json".freeze
s.licenses = ["Ruby".freeze]
s.rdoc_options = ["--title".freeze, "JSON implementation for Ruby".freeze, "--main".freeze, "README.md".freeze]
s.required_ruby_version = Gem::Requirement.new(">= 2.3".freeze)
s.rubygems_version = "3.2.33".freeze
s.summary = "JSON Implementation for Ruby".freeze
s.installed_by_version = "3.2.33" if s.respond_to? :installed_by_version
end
You're right, the issue is not when generating the gem package, but when installing it, since that gemspec
file with that "stub" comment is generated at gem install
time. To get the correct gemspec stub generated I believe you need to use gem install
with the --default
flag. However, that only installs the gemspec, not the rest of the gem files. So you're going to need some combination of both, or hack something else.
This is the difference in how the gemspecs are generated, depending on whether the --default
flag is passed or not: https://github.com/rubygems/rubygems/blob/629e08d34d14f60fe3398e40fc72f4f171ffc582/lib/rubygems/installer.rb#L452-L469.
Hm can I then just run gem install
twice? once with the flag and once without it? To me this sounds like a bug introduced in the json upstream code somewhere between 2.5.1 and 2.6.2
I don´t know how you're packaging things, but you need to make sure that the gemspec
file that gem install --default
installs is the one that ends up in the final Gem.default_specifications_dir
location. That file should include the full list of json
files, so that require
can detect them as files belonging to a default gem, and have special behaviour for them.
Running gem install --default --install-dir="tmp/"
and then copy the gemspec file from that folder to the proper place after a normal gem install
works.
With this fixed version of json
the tests actually progress and now I only have this one Failure and a few pending (because of unmet dependencies which is fine):
===============================================================================
Failure: test_build_extensions(TestGemExtBuilder): <"/build/rubygems/src/rubygems-3.3.23/tmp/test_rubygems_20221011-834-nt4d88/gemhome/gems/a-2/lib/a.rb"> was expected to exist
/build/rubygems/src/rubygems-3.3.23/test/rubygems/test_gem_ext_builder.rb:140:in `test_build_extensions'
137: assert_path_exist @spec.gem_build_complete_path
138: assert_path_exist File.join @spec.extension_dir, "gem_make.out"
139: assert_path_exist File.join @spec.extension_dir, "a.rb"
=> 140: assert_path_exist File.join @spec.gem_dir, "lib", "a.rb"
141: assert_path_exist File.join @spec.gem_dir, "lib", "a", "b.rb"
142: end
143:
===============================================================================
......F
===============================================================================
Failure: test_build_extensions_with_gemhome_with_space(TestGemExtBuilder): <"/build/rubygems/src/rubygems-3.3.23/tmp/test_rubygems_20221011-834-ghcwod/gem home/gems/a-2/lib/a.rb"> was expected to exist
/build/rubygems/src/rubygems-3.3.23/test/rubygems/test_gem_ext_builder.rb:140:in `test_build_extensions'
/build/rubygems/src/rubygems-3.3.23/test/rubygems/test_gem_ext_builder.rb:153:in `test_build_extensions_with_gemhome_with_space'
150: @spec = util_spec "a"
151: @builder = Gem::Ext::Builder.new @spec, ""
152:
=> 153: test_build_extensions
154: end
155:
156: def test_build_extensions_install_ext_only
===============================================================================
@deivid-rodriguez thanks for all the help so far. I have reported the json issue upstream ( https://github.com/flori/json/issues/505 ) as it seems to me that it is a problem in the json package itself.
Do you happen to know hoe i can fix these last two failures?
I don't see think this is an issue in the json gem. Generating gemspec stub files without the files =
attribute is expected, except for default gems. Only default gems need this attribute.
Regarding the other failures, I guess it could also be related to the same issue, since the --default
flag also has special behaviour for the extension dir: https://github.com/rubygems/rubygems/blob/629e08d34d14f60fe3398e40fc72f4f171ffc582/lib/rubygems/installer.rb#L306-L312.
Regarding the other failures, I guess it could also be related to the same issue, since the
--default
flag also has special behaviour for the extension dir:
So you are saying that other gems are affected by the same issue? If so is there any way to find out which one?
Any default gem you ship separately in the same way you ship json should be affected. You can find a list of official default gems here: https://stdgems.org.
I haven't looked at what the failures about the extension dir mean, but the other failures mean that RubyGems is not able to automatically activate the latest version of default gems on require.
What that means is the following. Say you have json-2.6.1 instead as a default gem, and json-2.6.2 is released with a security fix. With this feature of RubyGems, if you install gem install
json-2.6.2, then require "json"
will automatically activate and use json-2.6.2. Without this feature, require "json"
will always use the default version of json
.
Running
rake test
on versionv3.3.21
I get this result: