search

rubyjs / mini_racer

Minimal embedded v8
MIT License
594 stars 93 forks source link

Node release catch-up plan #277

Closed lloeki closed 5 months ago

lloeki commented 1 year ago

Current status

Proposed plan

Each intermediate version reduces risk towards a final release and provides a fallback spot to have an actual release with the widest OS and compiler support.

lloeki commented 1 year ago

16.19.0.1 has been built locally for all gem platforms and pushed to rubygems

lloeki commented 1 year ago

17.9.1.1 has been built locally for all gem platforms and pushed to rubygems

lloeki commented 1 year ago

18.13.0.1 has been built locally for all gem platforms and pushed to rubygems

SamSaffron commented 1 year ago

(2) bump mini_racer dependency requirement to ~> 16.19.0.0 and release mini_racer 0.6.4 bump release

This allows every user of current mini_racer to immediately move to a Node safer than 16.10.0 (and safest, short for one version)

Just tried to work through this but I am getting a segfault... 

-- C level backtrace information -------------------------------------------
/home/sam/.rubies/ruby-3.2.1/bin/ruby(rb_print_backtrace+0x14) [0x5597194692fb] /home/sam/src/ruby-3.2.1/vm_dump.c:785
/home/sam/.rubies/ruby-3.2.1/bin/ruby(rb_vm_bugreport) /home/sam/src/ruby-3.2.1/vm_dump.c:1080
/home/sam/.rubies/ruby-3.2.1/bin/ruby(rb_bug_for_fatal_signal+0xe8) [0x55971953de58] /home/sam/src/ruby-3.2.1/error.c:813
/home/sam/.rubies/ruby-3.2.1/bin/ruby(sigsegv+0x4b) [0x5597193b56bb] /home/sam/src/ruby-3.2.1/signal.c:964
/usr/lib/libc.so.6(0x7f6691a46ab0) [0x7f6691a46ab0]
/home/sam/.rubies/ruby-3.2.1/bin/ruby(RVALUE_MARKED+0x9) [0x559719276d03] /home/sam/src/ruby-3.2.1/gc.c:1656
/home/sam/.rubies/ruby-3.2.1/bin/ruby(gc_mark_set) /home/sam/src/ruby-3.2.1/gc.c:6930
/home/sam/.rubies/ruby-3.2.1/bin/ruby(gc_mark_ptr) /home/sam/src/ruby-3.2.1/gc.c:7044
/home/sam/.rubies/ruby-3.2.1/bin/ruby(rb_mark_end_proc+0x19) [0x559719255919] /home/sam/src/ruby-3.2.1/eval_jump.c:84
/home/sam/.rubies/ruby-3.2.1/bin/ruby(gc_mark_roots+0x3ed) [0x559719277efd] /home/sam/src/ruby-3.2.1/gc.c:7562
/home/sam/.rubies/ruby-3.2.1/bin/ruby(gc_marks+0x40d) [0x55971927af8d] /home/sam/src/ruby-3.2.1/gc.c:8244
/home/sam/.rubies/ruby-3.2.1/bin/ruby(gc_start) /home/sam/src/ruby-3.2.1/gc.c:9547
/home/sam/.rubies/ruby-3.2.1/bin/ruby(heap_prepare+0x22) [0x55971927ea0a] /home/sam/src/ruby-3.2.1/gc.c:2431
/home/sam/.rubies/ruby-3.2.1/bin/ruby(heap_next_free_page) /home/sam/src/ruby-3.2.1/gc.c:2672
/home/sam/.rubies/ruby-3.2.1/bin/ruby(newobj_alloc) /home/sam/src/ruby-3.2.1/gc.c:2780
/home/sam/.rubies/ruby-3.2.1/bin/ruby(newobj_of0+0x56) [0x55971927f669] /home/sam/src/ruby-3.2.1/gc.c:2876
/home/sam/.rubies/ruby-3.2.1/bin/ruby(newobj_of) /home/sam/src/ruby-3.2.1/gc.c:2896
/home/sam/.rubies/ruby-3.2.1/bin/ruby(rb_wb_protected_newobj_of) /home/sam/src/ruby-3.2.1/gc.c:2918
lloeki commented 1 year ago

19.9.0.0 has been built locally for all gem platforms and pushed to rubygems.

There are compilation failures when building mini_racer against it though.

lloeki commented 1 year ago

20.2.0.0 has been built locally for all gem platforms and pushed to rubygems.

There are compilation failures when building mini_racer against it though.

lloeki commented 1 year ago

Just tried to work through this but I am getting a segfault...

Strange, all tests were green before I pushed.

@SamSaffron Let's track it in a separate issue to keep this one focused on global progress

lloeki commented 1 year ago

@seanmakesgames would you like to test these gems out?

seanmakesgames commented 1 year ago

Yeah--

Yesterday middle of day filled up very quickly. Catching up now.

seanmakesgames commented 1 year ago

on mini_racer/master, updated LIBV8_NODE_VERSION = "~> 16.19.0.0"

ran bundle, ran rake clean compile test

2 skips, no failures

seanmakesgames commented 1 year ago

@SamSaffron

Just tried to work through this but I am getting a segfault...

What platform are you on for this one?

SamSaffron commented 1 year ago

Oh sorry about this... was a brainfart ... I forgot to clean ... all is good and new version is out

Updated @lloeki checklist!

SamSaffron commented 1 year ago

also merged in 17 :confetti_ball: but will let it live in the repo for a few days prior to cutting a release

tisba commented 1 year ago

Be aware that https://github.com/rubyjs/mini_racer/pull/271 merged set the version to 0.6.5, not to 0.7.0 as suggested above, see https://github.com/rubyjs/mini_racer/commit/7193406ff134e57e141eca5f042600d92edcd583#diff-57c74f5bd99f4a8425e28b076319f9935cd46a3b2bcfa6d6ccceddd651f2ce6fR4.

lloeki commented 1 year ago

Good catch @tisba!

@SamSaffron I'll cut:

lloeki commented 1 year ago

I have created stable-0.6 and stable-0.7, from which we'll be able to maintain and release older versions non-linearly (which is the case here).

Not that we usually should with any regularity, but having that mobility clearly defined is nice instead of comping up with ad-hoc solutions on the spot, especially when one has to do an emergency release. Also ties in with #276 as we should have a clearly delineated, written down support policy.

Another good side effect of these is that they can be referenced as is in libv8-node for running testing against mini_racer on the various node-XY branches.

lloeki commented 1 year ago
tisba commented 1 year ago

(will update this with further tests)

I'm going to test all supported Ruby verstions at their latest patch level (3.0.6, 3.1.4 and 3.2.2 currently) on M1 and x86 native on Ventura 13.4, as well as on aarch64-linux and x86-linux using Docker. For testing I'm using a minimal test (see below). For Ruby 3.2.2 I can run a more comprehensive product test suite (proprietary though).

RUBY_VERSION RUBY_PLATFORM branch minimal.rb proprietary
3.0.6 arm64-darwin22 stable-0.6 (16b521d) ✅
3.1.4 arm64-darwin22 stable-0.6 (16b521d) ✅
3.2.2 arm64-darwin22 stable-0.6 (16b521d) ✅ ✅
3.0.6 x86_64-darwin22 stable-0.6 (16b521d) ✅
3.1.4 x86_64-darwin22 stable-0.6 (16b521d) ✅
3.2.2 x86_64-darwin22 stable-0.6 (16b521d) ✅ ✅
3.0.6 aarch64-linux stable-0.6 (16b521d) ✅
3.1.4 aarch64-linux stable-0.6 (16b521d) ✅
3.2.2 aarch64-linux stable-0.6 (16b521d) ✅ ✅
3.0.6 arm64-darwin22 stable-0.7 (c0a6a34) ✅
3.1.4 arm64-darwin22 stable-0.7 (c0a6a34) ✅
3.2.2 arm64-darwin22 stable-0.7 (c0a6a34) ✅ ✅
3.0.6 x86_64-linux stable-0.6 (16b521d) ✅
3.1.4 x86_64-linux stable-0.6 (16b521d) ✅
3.2.2 x86_64-linux stable-0.6 (16b521d) ✅ ✅
3.0.6 aarch64-linux stable-0.7 (c0a6a34) ✅
3.1.4 aarch64-linux stable-0.7 (c0a6a34) ✅
3.2.2 aarch64-linux stable-0.7 (c0a6a34) ✅ ✅
3.0.6 arm64-darwin22 0.8.0 ✅
3.1.4 arm64-darwin22 0.8.0 ✅
3.2.2 arm64-darwin22 0.8.0 ✅ ✅
3.0.6 x86_64-darwin22 0.8.0 ✅
3.1.4 x86_64-darwin22 0.8.0 ✅
3.2.2 x86_64-darwin22 0.8.0 ✅
3.0.6 aarch64-linux 0.8.0 ✅
3.1.4 aarch64-linux 0.8.0 ✅
3.2.2 aarch64-linux 0.8.0 ✅ ✅
3.0.6 arm64-darwin22 0.8.0 ✅
3.1.4 arm64-darwin22 0.8.0 ✅
3.2.2 x86_64-linux 0.8.0 ✅ ✅
minimal.rb ```ruby # frozen_string_literal: true # Run with: # ruby minimal.rb # -or via docker- # docker run -it --rm -v "$(pwd)":/app -w /app ruby:3.2.2 ruby /app/minimal.rb require "bundler/inline" gemfile do source "https://rubygems.org" gem "mini_racer", github: "rubyjs/mini_racer", branch: "stable-0.7" end require "libv8-node" require "rbconfig" puts "RbConfig::CONFIG['LIBS']: #{RbConfig::CONFIG["LIBS"]}" puts "RUBY_VERSION : #{RUBY_VERSION}" puts "RUBY_PLATFORM: #{RUBY_PLATFORM}" puts "MiniRacer::VERSION: #{MiniRacer::VERSION}" puts "MiniRacer::LIBV8_NODE_VERSION: #{MiniRacer::LIBV8_NODE_VERSION}" puts "Libv8::Node::VERSION: #{Libv8::Node::VERSION}" puts "Libv8::Node::NODE_VERSION: #{Libv8::Node::NODE_VERSION}" puts "Libv8::Node::LIBV8_VERSION: #{Libv8::Node::LIBV8_VERSION}" ctx = MiniRacer::Context.new puts ctx.eval("1+1") ```
lloeki commented 1 year ago

the idea is to release:

then work on node 19, which should trickle down to making node 20 mostly (hopefully completely) work.

lloeki commented 1 year ago

Thanks @tisba for that testing, you can probably try branch stable-0.8 as well

SamSaffron commented 1 year ago

Yay... got releases out for 0.7.0 and 0.8.0 this morning :confetti_ball: thanks heaps!

tisba commented 1 year ago

Thanks @tisba for that testing, you can probably try branch stable-0.8 as well

Will try to put the released 0.8.0 through CI tomorrow. My "minimal" test seems fine.

tisba commented 1 year ago

Just tested 0.8.0 (see updated table above) and it looks all good.

Almost a little bit too smooth so far 😁

tisba commented 1 year ago

Looks like we're kind of stuck with our plan. Not sure what prevents us from step 8 (releasing 0.8.1) or why we need this step, but looking at https://github.com/rubyjs/mini_racer/pull/283 and https://github.com/rubyjs/mini_racer/issues/288 it looks like getting Node 19.x+ support will be a challenge.

Is there anything we can do in general so that we're not getting stuck (again) for a long time?

Fayti1703 commented 1 year ago

288 is not explicitly related to 19.x+ support, by the way -- the issue occurs in all versions.

lloeki commented 1 year ago

Thanks @Fayti1703 I confused it with this one: https://github.com/rubyjs/mini_racer/pull/283#issuecomment-1572735274

the issue occurs in all versions.

That may sound strange but if it's already there in a released version, I'm on the opinion that it should not be a blocker for a new release based on 19 or 20 (provided we fix new blockers). Of course we should ultimately find a way to fix it but I'd rather make sure we're catching up with the train.

lloeki commented 1 year ago

Not sure what prevents us from step 8 (releasing 0.8.1) or why we need this step

I don't think 0.8.1 is needed anymore as we jumped straight to node 18.16 on 0.8.0 via https://github.com/rubyjs/mini_racer/pull/261/commits/09783566c89bd1e68d65d4d301002441455df233

Looks like we're kind of stuck with our plan

Yeah that's basically in line with what I expected, and 19 is actually even more challenging than I anticipated. At least it's a stable target since it won't move anymore, and it should cover most of the ground towards 20 (well, except if https://github.com/rubyjs/mini_racer/pull/283#issuecomment-1585816428 is a node 19 bug fixed in node 20).

Nothing very effective comes to mind on the general front, the only way forward seems to be through it and get these crashers sorted out. FYI I'm booked for a couple of weeks while we kick off the quarter, then I can carve some time out to try to give @Fayti1703 a hand.

lloeki commented 6 months ago

I moved to try things on node 20 https://github.com/rubyjs/mini_racer/pull/284

tisba commented 6 months ago

Reading the initial description… it looks like we basically did Step 8

8) Bump mini_racer dependency requirement to ~> 18.16.0.0 and release mini_racer 0.8.1.

Only it has been released as 0.9.0, but 🤷 The 0.8.1 release works fine in my testing and we already adopted it to prod in my day job.

tisba commented 6 months ago

I moved to try things on node 20 #284

Just curious: Are you planning to skip V8 19 altogether, @lloeki? AFAIK #284 is still based on the [libv8-node-19](https://github.com/rubyjs/mini_racer/tree/libv8-node-19) branch.

In any case, let me know if I can help in any way like with testing things.

lloeki commented 6 months ago

it has been released as 0.9.0

Yeah @SamSaffron elected to release as 0.9.0.

Are you planning to skip V8 19 altogether

Possibly:

We'll see. It always was an optional step in the original plan.

is still based on the libv8-node-19

Yeah, not really a problem, e.g once 20 is fixed the libv8-node-20 PR could be merged to master instead (which would include the libv8-node-19 contents as historical transition) and the libv8-node-19 PR closed, or we could merge libv8-node-19 then libv8-node-20 (which would have its base automatically changed to master). Whatever works, but we're not there yet.

lloeki commented 6 months ago

In any case, to keep things up to date I've built and pushed:

libv8-node-20.12.1.0-aarch64-linux-musl.gem
libv8-node-20.12.1.0-aarch64-linux.gem
libv8-node-20.12.1.0-arm64-darwin.gem
libv8-node-20.12.1.0-x86_64-darwin.gem
libv8-node-20.12.1.0-x86_64-linux-musl.gem
libv8-node-20.12.1.0-x86_64-linux.gem
libv8-node-20.12.1.0.gem

And updated https://github.com/rubyjs/mini_racer/pull/284 to use that.

lloeki commented 6 months ago

On a whim I expended a few CPU cycles and pushed:

libv8-node-21.7.2.0-aarch64-linux-musl.gem
libv8-node-21.7.2.0-aarch64-linux.gem
libv8-node-21.7.2.0-arm64-darwin.gem
libv8-node-21.7.2.0-x86_64-darwin.gem
libv8-node-21.7.2.0-x86_64-linux-musl.gem
libv8-node-21.7.2.0-x86_64-linux.gem
libv8-node-21.7.2.0.gem

Guess what, https://github.com/rubyjs/mini_racer/pull/299 has all but one innocuous† test passing, no snapshot crash.

let me know if I can help in any way like with testing things.

@tisba I'm going to take you up to your words, can you test that libv8-node-21 branch?

† This is the same as https://github.com/rubyjs/mini_racer/pull/284#discussion_r1553476995 which might actually be a correct 0 value tripping an incorrect > 0 test.

lloeki commented 6 months ago

My updated core plan:

And stretch goals:

Rationale for the intermediate bumps and merges is to have valid branching points for stable branches should we elect to release mini_racer from these stable-* branches based on these versions. Notably a version based on node 20 has merits because it's LTS.

SamSaffron commented 6 months ago

wow this is fantastic news, thanks @lloeki

tisba commented 6 months ago

This is awesome news! 🚀

@tisba I'm going to take you up to your words, can you test that libv8-node-21 branch?

I can take some time in the next days to test things more thoroughly, but mini_racer from c30ec4dec3e9c2146effb740aacba6c8f98f23aa with libv8-node (21.7.2.0) looked already pretty good in my projects test suite (RUBY_PLATFORM: aarch64-linux, RUBY_VERSION : 3.3.0, Libv8::Node::VERSION: 21.7.2.0, Libv8::Node::LIBV8_VERSION: 11.8.172.17).

Update Same (☝️) on x86_64-linux ✅

Update 2 I did various simple sanity checks based on the same commit (☝️), all on arm64-darwin23 (macOS 14.4.1 (23E224)). Every currently non-EOL'd Ruby look fine: 3.1.4, 3.2.3 and 3.3.0 ✅

seanmakesgames commented 5 months ago

Shoot! We are so behind, we can't help test really. We have to update our ruby version and esprima first. Super pumped we're making some headway here though! Will help where we can.

Post your calls for support and we'll try!

lloeki commented 5 months ago

Thanks a lot folks! I'll proceed with the plan as time permits.

lloeki commented 5 months ago

total_heap_size_executable being 0 is legit, so I fixed the test to have V8 JIT stuff, sort of making sure it's > 0: https://github.com/rubyjs/mini_racer/commit/3754b179f9e86872a3f6bff0ecc3abe51d670474

lloeki commented 5 months ago

Waiting for CI, then I'll finally pull the trigger on 0.12.0, and then, upstream Node.js 21.7.3.0 having been released a dozen days ago, mini_racer will be the closest to "up to date" it has ever been since years.

lloeki commented 5 months ago

And it's released.

lloeki commented 5 months ago

Since we have caught up I'm closing this issue.

Stretch goals may or may not be attempted depending on whether there's any demand, in which case they should be tracked in their own issue.