search

oracle / truffleruby

A high performance implementation of the Ruby programming language, built on GraalVM.
https://www.graalvm.org/ruby/
Other
3.02k stars 185 forks source link

date-3.4.0/lib/date_core.bundle : symbol not found in flat namespace '_rb_str_format' #3716

Open tykonu opened 5 days ago

tykonu commented 5 days ago

Hey!

I wanted to experiment with transferring a large-ish Rails app to truffleruby, but it seems to fail every time when building the Nokogiri gem.

OS: MacOS Sequoia 15.1 (24B83) Ruby: truffleruby 24.1.1, like ruby 3.2.4, Oracle GraalVM Native [arm64-darwin20] (installed with rbenv) Rails: Tried 7.0, 7.1, 7.2 and 8.0

Tried different Nokogiri versions as well, still the same issue. Also tried truffleruby+graalvm-24.1.1, same issue.

It also fails when I install nokogiri separately with just gem install nokogiri

Could you help?

current directory: /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/nokogiri-1.16.7/ext/nokogiri
/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/bin/ruby extconf.rb --disable-static
checking for whether -std=c99 is accepted as CFLAGS... yes
checking for whether -Wno-declaration-after-statement is accepted as CFLAGS... yes
checking for whether -O2 is accepted as CFLAGS... yes
checking for whether -g is accepted as CFLAGS... yes
checking for whether -Winline is accepted as CFLAGS... yes
checking for whether -Wmissing-noreturn is accepted as CFLAGS... yes
checking for whether -Wshorten-64-to-32 is accepted as CFLAGS... yes
checking for whether -Wno-error=unused-command-line-argument-hard-error-in-future is accepted as CFLAGS... no
checking for whether -Wno-unknown-warning-option is accepted as CFLAGS... yes
Building nokogiri using packaged libraries.
Static linking is disabled.
Cross build is disabled.
checking for iconv.h... yes
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --without-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=.
    --curdir
    --ruby=/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/bin/ruby
    --help
    --clean
    --prevent-strip
    --enable-system-libraries
    --disable-system-libraries
    --use-system-libraries
    --enable-system-libraries
    --disable-system-libraries
    --use-system-libraries
    --enable-static
    --disable-static
    --enable-cross-build
    --disable-cross-build
    --enable-cross-build
    --disable-cross-build
    --with-zlib-dir
    --without-zlib-dir
    --with-zlib-include
    --without-zlib-include=${zlib-dir}/include
    --with-zlib-lib
    --without-zlib-lib=${zlib-dir}/lib
    --with-iconv-dir
    --without-iconv-dir
    --with-iconv-include
    --without-iconv-include=${iconv-dir}/include
    --with-iconv-lib
    --without-iconv-lib=${iconv-dir}/lib

Contents of mkmf.log:
block in append_cflags: checking for whether -std=c99 is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -o conftest -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args conftest.c  -L.  -L/opt/homebrew/opt/openssl@1.1/lib         -L/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext -Wl,-rpath,/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext -ltrufflerubytrampoline"
2024-11-11 14:56:07.441 xcodebuild[4900:43945] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionSentinelHostApplications for extension Xcode.DebuggerFoundation.AppExtensionHosts.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
2024-11-11 14:56:07.442 xcodebuild[4900:43945] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionPointIdentifierToBundleIdentifier for extension Xcode.DebuggerFoundation.AppExtensionToBundleIdentifierMap.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args  -std=c99 -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wno-declaration-after-statement is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99  -Wno-declaration-after-statement -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -O2 is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement  -O2 -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -g is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2  -g -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Winline is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g  -Winline -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wmissing-noreturn is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline  -Wmissing-noreturn -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wshorten-64-to-32 is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline -Wmissing-noreturn  -Wshorten-64-to-32 -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wno-error=unused-command-line-argument-hard-error-in-future is accepted as CFLAGS... -------------------- no

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline -Wmissing-noreturn -Wshorten-64-to-32  -Wno-error=unused-command-line-argument-hard-error-in-future -Werror -c conftest.c"
error: unknown warning option '-Werror=unused-command-line-argument-hard-error-in-future'; did you mean '-Werror=unused-command-line-argument'? [-Werror,-Wunknown-warning-option]
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wno-unknown-warning-option is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline -Wmissing-noreturn -Wshorten-64-to-32  -Wno-unknown-warning-option -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

have_header: checking for iconv.h... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/openssl@1.1/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline -Wmissing-noreturn -Wshorten-64-to-32 -Wno-unknown-warning-option   -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: #include <iconv.h>
/* end */

--------------------

<internal:core> core/kernel.rb:229:in `gem_original_require': dlopen(/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/date-3.4.0/lib/date_core.bundle, 0x0009): symbol not found in flat namespace '_rb_str_format' (RuntimeError)
    from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
    from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/date-3.4.0/lib/date.rb:4:in `<top (required)>'
    from <internal:core> core/kernel.rb:229:in `gem_original_require'
    from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
    from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/time-0.4.1/lib/time.rb:4:in `<top (required)>'
    from <internal:core> core/kernel.rb:229:in `gem_original_require'
    from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
    from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/open-uri.rb:4:in `<top (required)>'
    from <internal:core> core/kernel.rb:229:in `gem_original_require'
    from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:149:in `require'
    from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/mini_portile2-2.8.7/lib/mini_portile2/mini_portile.rb:7:in `<top (required)>'
    from <internal:core> core/kernel.rb:229:in `gem_original_require'
    from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
    from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/mini_portile2-2.8.7/lib/mini_portile2.rb:2:in `<top (required)>'
    from <internal:core> core/kernel.rb:229:in `gem_original_require'
    from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
    from extconf.rb:431:in `process_recipe'
    from extconf.rb:871:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/extensions/arm64-darwin/3.2.4.24.1.0.1/nokogiri-1.16.7/mkmf.log

extconf failed, exit code 1
eregon commented 5 days ago

I cannot reproduce on Linux (it works fine) with:

$ gem install date:3.4.0
$ ruby -e 'require "date"; puts $"'
$ gem install nokogiri

At the end of your output I can see you have date-3.4.0 installed:

<internal:core> core/kernel.rb:229:in `gem_original_require': dlopen(/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/date-3.4.0/lib/date_core.bundle, 0x0009): symbol not found in flat namespace '_rb_str_format' (RuntimeError)
    from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
    from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/date-3.4.0/lib/date.rb:4:in `<top (required)>'

And this sound like macOS is not respecting RTLD_LAZY, as in https://github.com/oracle/truffleruby/issues/3390 Do you have XCode 14.2? If not which version do you have?

FWIW, TruffleRuby does not implement rb_str_format() yet, but that's only used in date_strftime() -> date_strftime_with_tmx(), which is not called on require 'date' as far as I can tell so it seems to indicate that macOS is not respecting dlopen(path, RTLD_LAZY) here, or made it less lazy. In Init_date_core there is https://github.com/ruby/date/blob/v3.4.0/ext/date/date_core.c#L9811 which is not calling that but taking the reference of a function indirectly using rb_str_format(), maybe that's enough on macOS to trigger the error?

In any case, we should add TruffleRuby in ruby/date's CI and implement rb_str_format(), however the fact it works fine on Linux means macOS might have broken something in their linker/dynamic loader.

eregon commented 5 days ago

So just to clarify, this has nothing to do with nokogiri, it's about the date gem, how it uses rb_str_format() and how the dynamic loader handles undefined symbols.

tykonu commented 4 days ago

@eregon thank you for pointing me in the right direction. Not sure which Xcode version I had, but I updated it to 16.1 and the problem was solved.

Could you also confirm what's the latest version of Rails that is supported on 24.1.1? I noticed the bundle was installed fine on Rails 7.1+ but the app didn't start, it raised some NoMethodErrors related to ActionCable 7.1+. Downgraded to Rails 7.0 and ActionCable 7.0 and the app launched correctly.

eregon commented 4 days ago

it raised some NoMethodErrors related to ActionCable 7.1+

Could you share those NoMethodErrors as a separate issue?

Generally any version of Rails should work, currently known issues are https://github.com/oracle/truffleruby/issues/3683#issuecomment-2424022506 for Zeitwerk (applies to multiple Rails versions) and https://github.com/rails/rails/pull/53490#issuecomment-2465235043 (Rails 8).

andrykonchin commented 1 day ago

Just for the record.

Now there are the following failures in the date gem tests on TruffleRuby:

1)

`Failure: test_to_time__from_datetime(TestDateConv)
.../date/test/date/test_date_conv.rb:95:in `test_to_time__from_datetime'
     92:     if Time.allocate.respond_to?(:subsec)
     93:       d = DateTime.new(2004, 9, 19, 1, 2, 3, 0) + 456789123456789123.to_r/86400000000000000000000
     94:       t = d.to_time.utc
  => 95:       assert_equal([2004, 9, 19, 1, 2, 3, Rational(456789123456789123,1000000000000000000)],
     96:           [t.year, t.mon, t.mday, t.hour, t.min, t.sec, t.subsec])
     97:     end
     98:   end
<internal:core> core/throw_catch.rb:36:in `catch'
<internal:core> core/throw_catch.rb:36:in `catch'
<[2004, 9, 19, 1, 2, 3, 456789123456789123/1000000000000000000]> expected but was
<[2004, 9, 19, 1, 2, 3, 456789123/1000000000]>

2)

Failure: test__parse_too_long_year(TestDateParse)
.../date/test/date/test_date_parse.rb:594:in `test__parse_too_long_year'
     591:   def test__parse_too_long_year
     592:     str = "Jan 1" + "0" * 100_000
     593:     h = EnvUtil.timeout(3) {Date._parse(str, limit: 100_010)}
  => 594:     assert_equal(100_000, Math.log10(h[:year]))
     595:     assert_equal(1, h[:mon])
     596:
     597:     str = "Jan - 1" + "0" * 100_000
<internal:core> core/throw_catch.rb:36:in `catch'
<internal:core> core/throw_catch.rb:36:in `catch'
<100000> expected but was
<Infinity>

The reason of the first failure is that Time.new (that is used by DateTime#to_time) on TruffleRuby truncates seconds fraction to nanoseconds, so only 9 digits are stored. That's why 456789123456789123/1000000000000000000 becomes 456789123/1000000000. On CRuby Time instance stores not truncated subseconds.

The second failure occurs because on TruffleRuby Math.log10 accepts only Java double argument. And 10^100_000 value exceeds double's range of 4.9e-324 to 1.7e+308.

So it seems to we can just disable these tests on TruffleRuby.