Closed eregon closed 3 years ago
Although bootsnap
would be nice it's not critical in transitioning production apps from MRI to TruffleRuby. For my case having database gems like pg
and mysql2
would be the most beneficial by far.
bootsnap
can easily be made compatible by disabling a couple optimization like it's already done for JRuby. The real blocker is that it depends on msgpack
. But I'm tracking this issue and whenever msgpack
is supported I can fix bootsnap
.
@skunkworker Indeed, the table is in no particular order but pg
/mysql2
/nokogiri
are the most important right now out of this list.
@byroot That's great to hear. I did a bit of work to get basics of msgpack
running some time ago, I'll try to progress on that.
@eregon could Truffle use JDBC adapters as a workaround for pg/mysql ? perhaps related: https://github.com/pgjdbc/pgjdbc/issues/1189
@paneq TruffleRuby on the GraalVM should be able to support JDBC, as we have a rich Java interop API. Having said that, I don't think anyone on our team has spent any considerable amount of time looking into that.
Assuming it does work, it'd be a fine option for making JDBC calls from Ruby. Unfortunately, the use case we're really after is Rails. Unlike many other languages, Ruby does not have a DB abstraction layer as part of its standard library. Consequently, ActiveRecord ties very closely to the native mysql & pg drivers. This is something that has been a very big problem for the JRuby team. They provide an implementation of ActiveRecord based on JDBC, but since ActiveRecord's internals are subject to change, this breaks with every minor Rails release. And since ActiveRecord doesn't have a real plugin API, any ActiveRecord plugin you see really is a monkeypatch to the internals and these often don't work with the JRuby implementation.
A secondary concern is the binary available via all the Ruby version installers does not include Java interop support at the moment. So you could not use JDBC in that case either.
This is all a roundabout way of saying that working with JDBC for the use cases most Rubyists would want a DB for would entail quite a bit of effort. And we'd rather allocate that effort to supporting the native extensions, which would lead to a much smoother usage path.
@zw963 I think we run Sequel itself, but Sequel depends on pg
, mysql
, etc for actually talking to databases, so we need to support these C extensions anyway.
There are some pure-Ruby database drivers like postgres-pr, but those have a different API, won't work with ActiveRecord, and so not really a good workaround.
Sequel for Roda would be important via the pg
& mysql
support, yes. I think these are pretty good starts. With the recent deplorable announcement by GitHub to move away from Ruby & Rails since the MS acquisition, I think some heat is on. I've heard of some serious debates at workplaces coming up again against Ruby and Truffle is becoming more important than ever if it can reach a culminating point of running Rails out of the box and with minimal extra effort (such as special conditions for nokogiri
). Humans are very social and shifty creatures that rely on mass acceptance and good feelings often rather than fact and marketing, hate to say it, is key.
deplorable announcement by GitHub to move away from Ruby & Rails since the MS acquisition, I think some heat is on
@ylluminate Could you provide a link?
@zw963 it appears as though pg 1.0.0 can be compiled successfully, but versions 1.1.0, 1.1.1, and 1.1.2 cannot. Is this an error you all have seen? If not, I can paste the compile error here or file a new issue.
Update: This comment is replaced by issue #1421
@sbellware Could you file this as a separate issue? This issue is for keeping track of general progress, not individual compilation/running issues.
@eregon Done. The new issue is #1421.
@byroot
bootsnap
can easily be made compatible by disabling a couple optimization like it's already done for JRuby. The real blocker is that it depends onmsgpack
. But I'm tracking this issue and whenevermsgpack
is supported I can fixbootsnap
.
In TruffleRuby 1.0.0-rc6, gem install msgpack
succeeds, but require "msgpack"
fails.
Is msgpack
necessary to run bootsnap or is it more like an optional dependency?
We'll probably get msgpack
working in the future, but it would be useful to know if bootsnap needs it or it could be made optional easily for TruffleRuby.
Is
msgpack
necessary to run bootsnap or is it more like an optional dependency?
Kind of both. In short bootsnap need to be able to serialize a big hash efficiently, and messagepack shines here.
In theory we could fallback to use JSON or whatever, but it would complexify the codebase, and slow boot down significantly.
Alternatively we can probably turn bootsnap off entirely for truffleruby.
@byroot Thanks for the reply. Just an idea, would using Marshal
(which is part of the core library) instead work well? Or is it slower than msgpack to serialize that big hash?
Or is it slower than msgpack to serialize that big hash?
It's between 4 and 5 times slower. It's also much less compact.
@byroot Thanks for the info, we'll look at getting msgpack to work then.
I've been watching this project eagerly for a while as it matures, hoping that it will let users of Redmine (a popular project-management tool) get better performance across the board. I found some references to Redmine in the TruffleRuby GitHub repo, so I'm hoping the talented folks creating maintaining this project will have a vested interest in supporting Redmine.
Please let me know if any information is missing and I'll be glad to post more detail. I posted separate bug reports per @eregon's request, see below.
@JongleurX Could you file that report as a separate issue? Otherwise, it is very hard to track. I tried to clarify at the top of this issue that this issue is only a summary, and not where bugs should be reported.
Sure, I've reported separate issues for each gem that generated errors, and another separate one for the two rake failures. See the links above (I edited my original comment).
@JongleurX A single new issue is fine actually, since the last two errors seem related, and the opt fail should be fixed in the upcoming release (see #1422).
Perhaps ethon
should be added to this list?
Related issue: https://github.com/oracle/truffleruby/issues/1524
Typhoeus, one of the best performing http adapters, depends on this.
@dbackeus ethon
uses FFI, it's not a C extension. I commented a bit more about it on its issue #1524.
@byroot msgpack
now passes all specs on TruffleRuby, and it will be in the next release.
Awesome. Then I'll try to make bootsnap compatible. I suppose I can use truffle-head
or similar ?
@byroot Currently, you can either compile from source or wait for the next release (should be beginning of February, probably easier, I'll ping here when it's there). We also plan to have nightly builds in the future (https://github.com/oracle/truffleruby/issues/1483).
👌
I still can start now by adding minimal support for jruby & truffle, i.e. make bootsnap a noop on non MRI.
Then we can attempt to add load path cache support later on, but even if we don't at least people won't have to comment out bootsnap to try truffle.
@byroot That seems a good starting point indeed. Please ping me on the bootsnap PR.
PR opened: https://github.com/Shopify/bootsnap/pull/221
I'd like to add the popular grpc
gem to this list.
@pojntfx Could you file a new issue with the error you see?
I'll close this issue because all C extensions listed in the description work now :tada:, except mini_racer
which is tracked in #1827.
This issue summarizes C extensions not yet supported by TruffleRuby. Issues for failing C extensions should be filed separately as a new issue.
Note that we intend to support most C extensions, but we need to prioritize.
Not yet supported
mini_racer
#1827Works with workarounds
nokogiri
#62bundle config build.nokogiri --use-system-libraries
/gem install nokogiri -- --use-system-libraries
. See their documentation for details.Works out of the box
bootsnap
msgpack
mysql2
#1388pg
#1421hiredis
#1546 #1974rugged
#1088 #1642rbtree
#1343oj
#1566json
stdlib.oj
>= 2.7.1 installs, but does not run yet.And many more:
openssl
,psych
,zlib
,syslog
,etc
,puma
,sqlite3
,unf
, ...