Can't install 3.4.0-preview1

fallwith commented 3 months ago

Description

I am unable to install ruby v3.4.0-preview1 with ruby-install.

Steps To Reproduce

Using ruby-install v0.9.3 produces an error:

RUBY_CONFIGURE_OPTS="--enable-yjit" ruby-install 3.4.0-preview1 -- --with-opt-dir="$(brew --prefix openssl@3):$(brew --prefix readline):$(brew --prefix libyaml):$(brew --prefix gdbm)"

...

compiling yjit.c
building Rust YJIT (release mode)
touch yjit/target/release/libyjit.a
partial linking yjit/target/release/libyjit.a into yjit/target/release/libyjit.o
make: *** No rule to make target `coroutine/amd64/Context.S', needed by `coroutine/amd64/Context.o'.  Stop.
!!! Compiling ruby 3.4.0-preview1 failed!

manually building from source works:

RUBY_CONFIGURE_OPTS="--enable-yjit" ./configure --with-opt-dir="$(brew --prefix openssl@3):$(brew --prefix readline):$(brew --prefix libyaml):$(brew --prefix gdbm)" --prefix="$HOME/.rubies/ruby-3.4.0-preview1"
make
make install

Expected Behavior

Expected no issues and an installed instance of Ruby 3.4.0-preview1, as ruby-install has been fantastic with 3.3.1, 3.3.0, and the 3.3.0-previewN releases recently.

Actual Behavior

I don't end up with a 3.4.0-preview1 instance.

make: *** No rule to make target `coroutine/amd64/Context.S', needed by `coroutine/amd64/Context.o'.  Stop.
!!! Compiling ruby 3.4.0-preview1 failed!

Environment

$ ruby-install --version
[ruby-install v0.9.3](ruby-install: 0.9.3)

$ uname -a
Darwin C02GK4HNMD6P 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:28:58 PST 2023; root:xnu-10002.81.5~7/RELEASE_X86_64 x86_64

$ cc --version
Apple clang version 15.0.0 (clang-1500.1.0.2.5)
Target: x86_64-apple-darwin23.3.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

$ $(brew --prefix openssl@3)/bin/openssl --version
OpenSSL 3.3.0 9 Apr 2024 (Library: OpenSSL 3.3.0 9 Apr 2024)

$ rustc --version
rustc 1.78.0 (9b00956e5 2024-04-29)

postmodern commented 3 months ago

ruby-install does not use RUBY_CONFIGURE_OPTS, that is an environment variable from ruby-build. Instead try running ruby-install ... -- --enable-yjit. Also explicitly passing --with-opt-dir has not been necessary for a while; ruby-install-0.9.3 even added support for automatically passing in --with-openssl-dir in addition to --with-opt-dir.

Did manual building succeed or fail? If manual building failed, then this is an upstream issue that should be reported to bugs.ruby-lang.org.

havenwood commented 3 months ago

Just a side note, but YJIT will be installed with modern Rubies even without an explicit --enable-yjit flag as long as Rust is available.

fallwith commented 3 months ago

Thanks @postmodern and @havenwood.

Manually building from source works with the commands listed above in the "Steps To Reproduce" section. The make and make install commands both succeed and leave me with a usable copy of v3.4.0-preview1:

$ ~/.rubies/ruby-3.4.0-preview1/bin/ruby -v
ruby 3.4.0preview1 (2024-05-16 master 9d69619623) +YJIT [x86_64-darwin23]

For the ruby-install approach, I tried removing all of the options entirely and still encounter the same error:

$ ruby-install 3.4.0-preview1

(additional output was here)

compiling prism/util/pm_strncasecmp.c
compiling prism/util/pm_strpbrk.c
compiling prism/prism.c
compiling prism_init.c
compiling yjit.c
building Rust YJIT (release mode)
touch yjit/target/release/libyjit.a
partial linking yjit/target/release/libyjit.a into yjit/target/release/libyjit.o
make: *** No rule to make target `coroutine/amd64/Context.S', needed by `coroutine/amd64/Context.o'.  Stop.
!!! Compiling ruby 3.4.0-preview1 failed!

postmodern commented 3 months ago

@fallwith I'm curious what happens if you try doing make clean or deleting and re-extracting the source, then running the manual Steps To Reproduce. I suspect by manually going into the source code and running ./configure, make, make install, a second time, it's somehow creating the needed file?

fallwith commented 3 months ago

@postmodern, it looks like you're onto something with make clean. It removes the file in question and once it's missing I can reproduce the error while manually working with the Ruby source. I discovered the following:

The coroutine/amd64/Context.S file exists in the ruby-3.4.0-preview1.tar.gz source file to begin with
make clean cannot be ran until ./configure has been ran
Running make clean removes the coroutine/amd64/Context.S file
Running ./configure again will not recreate the coroutine/amd64/Context.S file
Running make with the coroutine/amd64/Context.S file missing recreates the same error I'm encountering with ruby-install

Using the Ruby 3.4.0-preview1 source (without any ruby-install involvement):

$ tar xzf ruby-3.4.0-preview1.tar.gz 

$ cd ruby-3.4.0-preview1 

$ ll coroutine/amd64 
.rw-r--r--@ 2.2k 501 20 2024-05-15 21:54 Context.h
.rw-r--r--@ 1.1k 501 20 2024-05-15 21:54 Context.S

$ ./configure

$ ll coroutine/amd64 
.rw-r--r--@ 2.2k 501 20 2024-05-15 21:54 Context.h
.rw-r--r--@ 1.1k 501 20 2024-05-15 21:54 Context.S

$ make clean

$ ll coroutine/amd64
.rw-r--r--@ 2.2k 501 20 2024-05-15 21:54 Context.h

$ ./configure

$ ll coroutine/amd64
.rw-r--r--@ 2.2k 501 20 2024-05-15 21:54 Context.h

$ make

(lots of healthy output here)

compiling yjit.c
building Rust YJIT (release mode)
touch yjit/target/release/libyjit.a
partial linking yjit/target/release/libyjit.a into yjit/target/release/libyjit.o
make: *** No rule to make target `coroutine/amd64/Context.S', needed by `coroutine/amd64/Context.o'.  Stop.

postmodern commented 3 months ago

@fallwith This sounds like a flaw in CRuby's Makefile or configure script logic. If make clean is going to remove the file, then either the Makefile should have a task to generate it, or configure should handle creating it. If there's no code which can re-generate the file, then make clean should probably not delete it.

fallwith commented 3 months ago

@postmodern I agree completely. ruby-install's performing of make clean is revealing a flaw in Ruby v3.4.0-preview1 itself and no changes should be made in the ruby-install repo. I'll close this issue and start a new one with ruby/ruby.

fallwith commented 3 months ago

I created Bug #20495 against Ruby.

fallwith commented 3 months ago

@postmodern here is the response I received:

For now, make clean is for those who "git clone" the repository. Don't use it for tarball package.

Thoughts?

postmodern commented 3 months ago

@postmodern here is the response I received:

For now, make clean is for those who "git clone" the repository. Don't use it for tarball package.

Thoughts?

That seems odd. make clean should always delete any built artifacts; with the caveat that make can rebuild them. They should probably add another make distclean or make gitclean task.

The reason why we added make clean before calling ./configure was to deal with re-compiling a previously installed ruby, but with different options. We also keep the source code directory in case you want to run ruby under gdb, in which case gdb will display the actual source code lines for functions.

We could add some kind of check whether bin/ruby exists and then run make clean, but that seems like we're working around an upstream flaw; and it wouldn't catch when a ruby source directory was half-compiled.

fallwith commented 2 months ago

@postmodern the underlying issue has been fixed with https://github.com/ruby/ruby/pull/10699. Thanks for the fix and for chiming in on the Ruby bug report, @eileencodes! So hopefully things will work fine with 3.4.0-preview2 once it is released.

postmodern / ruby-install