Open headius opened 1 day ago
Re RUBY_PLATFORM, it would be nice if JRuby would be compatible there, and be the native platform, i.e. like the [...]
part in RUBY_DESCRIPTION: https://github.com/jruby/jruby/issues/6152
Because it sounds like this workaround is and will be needed in many places.
(and on top of that JRuby docs suggest/ed to use RbConfig::CONFIG['host_os']
but apparently that's the wrong one, and should be target_os
, I recall some discussion about this on the FFI & Ruby tracker).
Re not finding the library, I suspect that's because JRuby uses a rather old FFI which doesn't have this fix to search for Homebrew on darwin-arm64: https://github.com/ffi/ffi/pull/968 Or that your Homebrew is in a non-standard location maybe.
it would be nice if JRuby would be compatible there
JRuby has used "java" for RUBY_PLATFORM for almost twenty years and there are hundreds of gems out there that expect it to remain that way, as I described in jruby/jruby#6152.
We originally were forced to use "java" here because RubyGems could not support JRuby-specific extension gems any other way. I'm not sure when RubyGems switched to RbConfig::CONFIG['arch']
but in any case the cat was out of the bag long before the other Ruby implementations even existed. Claiming "java" as our platform also made sense for many other reasons, and it has long been JRuby's goal that applications work the same on Windows as on Unix.
I suspect that's because JRuby uses a rather old FFI
JRuby does not use a "rather old FFI". JRuby sources all FFI Ruby code from the gem, currently at version 1.16.3. That fix appears to have been released in 1.16.0.
That should be updated to 1.17.0 but the fix you describe should already be shipping with JRuby.
Same result with FFI 1.17.0:
$ jruby -e 'require "ffi"; puts $".grep(/ffi.rb/); require "reline"'
/Users/headius/work/jruby/lib/ruby/gems/shared/gems/ffi-1.17.0-java/lib/ffi/ffi.rb
/Users/headius/work/jruby/lib/ruby/gems/shared/gems/ffi-1.17.0-java/lib/ffi.rb
LoadError: Could not open library 'libncursesw.dylib' : dlopen(libncursesw.dylib, 0x0009): tried: 'libncursesw.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibncursesw.dylib' (no such file), '/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/./libncursesw.dylib' (no such file), '/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/../lib/libncursesw.dylib' (no such file), '/usr/lib/libncursesw.dylib' (no such file, not in dyld cache), 'libncursesw.dylib' (no such file), '/usr/lib/libncursesw.dylib' (no such file, not in dyld cache)
open at org/jruby/ext/ffi/jffi/DynamicLibrary.java:100
initialize at /Users/headius/work/jruby/lib/ruby/gems/shared/gems/fiddle-1.1.3/lib/fiddle/ffi_backend.rb:478
new at org/jruby/RubyClass.java:921
curses_dl at /Users/headius/work/jruby/lib/ruby/stdlib/reline/terminfo.rb:41
each at org/jruby/RubyArray.java:1954
curses_dl at /Users/headius/work/jruby/lib/ruby/stdlib/reline/terminfo.rb:40
<main> at /Users/headius/work/jruby/lib/ruby/stdlib/reline/terminfo.rb:152
require at org/jruby/RubyKernel.java:1186
require at /Users/headius/work/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:136
<main> at /Users/headius/work/jruby/lib/ruby/stdlib/reline.rb:9
require at org/jruby/RubyKernel.java:1186
require at /Users/headius/work/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:136
<main> at -e:1
Again, as I said in the original description, this might be a JRuby issue, but I am filing it here for other reasons:
Perhaps there's a way to make this dependency on libncurses lazy and only load and use it if necessary? It's a separate issue from the inability to load on MacOS (or specifically on my machine), but it is definitely a concern for e.g. minimized Docker containers that may not need libncurses otherwise.
JRuby has used "java" for RUBY_PLATFORM for almost twenty years
Another point: RubyGems decided it was better to use RbConfig::CONFIG['arch']
to select an appropriate native extension gem. Why isn't that (or similar) the right way to locate native dependencies in libraries like reline?
What's the path of your libncurses.dylib
in Homebrew?
Are you on macos-arm64 or macos-amd64?
If the former, /opt/homebrew/lib
should be shown as part of the error message, that's why I thought too old FFI might be the issue.
Maybe some platform detection goes wrong on JRuby? The logic on master is https://github.com/ffi/ffi/blob/c128cede750242fe19945af8bd6c797728489ad5/lib/ffi/dynamic_library.rb#L37-L49
ncurses appears to install in /opt/homebrew/Cellar
:
$ find /opt/homebrew -name libncurses\*dylib
/opt/homebrew/Cellar/ncurses/6.5/lib/libncurses.6.dylib
/opt/homebrew/Cellar/ncurses/6.5/lib/libncurses++w.6.dylib
/opt/homebrew/Cellar/ncurses/6.5/lib/libncursesw.dylib
/opt/homebrew/Cellar/ncurses/6.5/lib/libncurses.dylib
/opt/homebrew/Cellar/ncurses/6.5/lib/libncursesw.6.dylib
/opt/homebrew/Cellar/ncurses/6.5/lib/libncurses++.dylib
/opt/homebrew/Cellar/ncurses/6.5/lib/libncurses++w.dylib
/opt/homebrew/Cellar/ncurses/6.5/lib/libncurses++.6.dylib
$ brew install ncurses
...
Warning: ncurses 6.5 is already installed and up-to-date.
To reinstall 6.5, run:
brew reinstall ncurses
I am on macos-aarch64. The SEARCH_PATH does appear to be set up with /opt/homebrew/lib but I do not know why it doesn't show in the error message:
[] json $ jruby -rffi -e 'p FFI::Platform::ARCH'
"aarch64"
[] json $ jruby -rffi -e 'p FFI::Platform.mac?'
true
[] json $ jruby -rffi -e 'p FFI::DynamicLibrary::SEARCH_PATH'
["/opt/homebrew/lib", "/opt/local/lib", "/usr/local/lib", "/usr/lib"]
Neither does /opt/local/lib
, or /usr/local/lib
.
Mmh, so there is no /opt/homebrew/lib/ncurses*.dylib
? It seems not with that find
output.
Looking at https://github.com/Homebrew/homebrew-core/blob/a320ea608f4eef1318e12c3a5afeaa0a080858f3/Formula/n/ncurses.rb#L22 so it is keg-only and not linked under /opt/homebrew/lib
.
The reason is "provided_by_macos", does macOS ship libncurses?
There are two issues to address here:
The reason is "provided_by_macos", does macOS ship libncurses?
I have not been able to find it in the usual places, but macOS sprinkles libraries all over the place.
macOS sprinkles libraries all over the place
I have been unable to locate libncurses.dylib anywhere in the standard macOS dirs.
This may be related (.tbd as a new dylib extension for "text-based stub libraries"): https://discourse.cmake.org/t/newer-versions-of-macos-require-linking-against-tbd-files-and-not-old-system-paths-to-dylib/6871/2
I do have some libncurses*.tbd
files in various places.
tbd files for libncurses on my machine. Note that without xcode these would not exist either. I can find no evidence that macOS provides libncurses in any reliable way (and there seems to be some debate about this elsewhere online too):
/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libncurses.5.tbd
/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libncurses.5.4.tbd
/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libncurses.tbd
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/lib/libncurses.5.tbd
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/lib/libncurses.5.4.tbd
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/lib/libncurses.tbd
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libncurses.5.4.tbd
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libncurses.tbd
@headius If you look at https://github.com/ruby/reline/blob/master/lib/reline/terminfo.rb libcurses is clearly optional. You're getting a LoadError above, but it should be rescued by https://github.com/ruby/fiddle/blob/1f818e46843965fbbb085114f0b875c3acf87489/lib/fiddle/ffi_backend.rb#L478-L479
Ah it's a bug of that code:
@lib = FFI::DynamicLibrary.open(libname, flags) rescue LoadError
is the same as
@lib = begin
FFI::DynamicLibrary.open(libname, flags)
rescue
LoadError
end
I'll fix it in ruby/fiddle: https://github.com/ruby/fiddle/pull/156
Hello!
We recently started pulling the latest stdlib gems into JRuby for Ruby 3.4 support and ran into an issue with reline loading libncurses via fiddle:
Several locations are searched here but the file is not found. A few possible issues:
.dylib
name rather than.so
.RUBY_PLATFORM
isjava
and RbConfig should be used to get the real host platform.libncurses.dylib
installed via Homebrew.We are keen to figure this issue out so we can proceed with running the latest gems, stdlib, and tests in JRuby's 3.4-compatible branch.
Patch for reline to properly select a set of libncurses file names: