Heroku environment

[MikaelMollberg]

Hi, I tried installing rice on heroku but received this error.

remote: extconf.rb:23:in

': Unfortunately Rice does not work against a Ruby without any shared libraries. (RuntimeError) remote: You'll need to rebuild Ruby with --enable-shared to use this library.

I contacted their support and they said that ruby is built with --enable-shared and told me to contact the maintainer. Any idea what could be wrong?

According to them Ruby is compiled with the following flags.

'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=/usr/lib/ruby1.9.1' '--srcdir=.' '--disable-maintainer-mode' '--disable-dependency-tracking' '--disable-silent-rules' '--enable-pthread' '--enable-shared' '--disable-rpath' '--disable-install-doc' '--with-vendordir=/usr/lib/ruby/vendor_ruby' '--with-sitedir=/usr/local/lib/site_ruby' '--program-suffix=1.9.1' '--with-soname=ruby-1.9.1' '--enable-ipv6' '--with-dbm-type=gdbm_compat' '--with-tklib=tk8.5' '--with-tcllib=tcl8.5' '--with-tcl-include=/usr/include/tcl8.5' '--with-tk-include=/usr/include/tcl8.5' '--with-tcl-lib=/usr/lib/x86_64-linux-gnu' '--with-tk-lib=/usr/lib/x86_64-linux-gnu' '--with-bundled-sha1' '--with-bundled-md5' '--with-bundled-rmd160' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall -fno-strict-aliasing' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -L/build/buildd/ruby1.9.1-1.9.3.484/debian/lib' 'CPPFLAGS=-D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall -fno-strict-aliasing'

[jasonroelofs]

That's odd. It's possible that the way the build system is checking for shared libraries is no longer valid.

What does the following give you in a heroku console?

require 'rbconfig'
RbConfig::CONFIG["ENABLE_SHARED"]

[MikaelMollberg]

irb(main):001:0> require 'rbconfig' => false irb(main):002:0> RbConfig::CONFIG["ENABLE_SHARED"] => "yes"

[jasonroelofs]

That's strange. That error shouldn't be hitting then if you get "yes" (https://github.com/jasonroelofs/rice/blob/master/extconf.rb#L22). That would imply that the tool that builds your Heroku slug has a different Ruby setup than the Dyno. Or maybe there's something else going on here?

[schneems]

Hey, I maintain the Ruby buildpack. We use the same image for building rubies that we use for running in production.

I've got limited experience with c-extensions, but maybe I can help. What exactly is Rice trying to do when it's getting this exception. Is the error coming from trying to include a header or something? Maybe one of the path's is different/missing ? Can you think of anything we can do to trouble shoot or dig deeper into this error?

[jasonroelofs]

@schneems Hey! Rice doesn't work against statically linked Ruby builds so in one of the first things we do in extconf.rb is to see if Ruby has a shared library built using the RbConfig struct. I've linked to the line in question in my comment previous to yours. For some reason, that value is returning "no" for @MikaelMollberg when trying to build the slug.

@MikaelMollberg what version of Ruby are you using in your app?

[MikaelMollberg]

I am using ruby 2.2.1p85

[jasonroelofs]

@MikaelMollberg Thanks. That then doesn't match up with the build flags Heroku gave you, as some of those flags reference files and libraries used for ruby-1.9.

[Kagetsuki]

Wait, has anyone actually been able to use rice on Heroku before? As far as I know it's never worked on heroku.

Basically to use a native gem on Heroku you need to either have a pre-compiled native available on rubygems or you need to use a Cedar 14 compatible environment and gem-compiler to compile the gem, then unpack it and vendorize it ALONG WITH librub.so/libruby.so.2.2/libruby.so.2.2.0 and any shared libraries your're linking against.

BUT! There is a reason I am here now and that reason is there seems to be some sort of issue with any [static] ruby past 2.2.0. From 2.2.1 on gems using rice, both pre-compiled gems on rubygems and vendorized versions don't work. They throw "LoadError: incompatible library version" because they can't link to libruby.so. I don't think this is a Heroku issue so much as it is a flaw of statically compiled ruby in 2.2.1 on.

@schneems If we could get dynamic ruby on Heroku this would be fixed. I tested this on a cedar:14 Docker instance and know this to be true. If we can't get dynamic ruby then the only solution I can think of is to find some way to get rice to work with static ruby - but I don't see that being any sort of easy solution ( @jasonroelofs can you confirm getting rice to work on static ruby is likely non-viable? ).

@jasonroelofs

For some reason, that value is returning "no"

Heroku is only using static ruby, returning "no" is accurate. Previously (2.2.0 and below) simply vendorizing and including the location of libruby.so[.*] in the LD_LIBRARY_PATH overrode this but from 2.2.1 this seems to not be the case. As a side note, both 2.2.0 and 2.2.3 return "no" when checking ENABLE_SHARED but 2.2.0 still loads shared libs if you have them available.

EDIT: @jasonroelofs you said:

That's odd. It's possible that the way the build system is checking for shared libraries is no longer valid.

I think for 2.2.1+ this is the issue.

[Kagetsuki]

@schneems Straight question: what's the viability of getting a dynamic ruby buildpack? Is that doable or is there some issue?

[schneems]

Sorry for dropping off the issue. All my github replies were going to my trash.

I'm not directly in charge of building binaries (in the past). This is what we've been using for our build process https://github.com/hone/docker-heroku-ruby-builder. There was a concerted effort to statically compile the Ruby libraries, it solved highly problematic issues, however I wasn't building rubies for the platform at that time so I don't know what the issues were.

[Kagetsuki]

@schneems I'm actually building an isolated dynamic ruby 2.2.3 on the cedar:14 Docker image and I was going to kludge that into a test app to play with it. I'd imagine if you tried really heard you could also just use the multi buildpack and grab the brightbox rubies. I don't know if either of these would work but if either of them would then I'd imagine it'd light up your dyno manager with whatever issue you were trying to avoid.

Do you think you can track down some info on why dynamic was a no-go? I wonder if whatever issues existed have been resolved.

[Kagetsuki]

@hone I'm totally dragging you into this because you seem to be a major player in the MRI build for Heroku and because you are awesome. Do you think a dynamic ruby build would be viable? Are you aware of what the issues are with dynamic ruby on Heroku?

[schneems]

He's on vacation right now.

[jasonroelofs]

@Kagetsuki The reason Rice won't work with a static build of Ruby is because Rice itself builds a linkable object (librice.a) that the final extension will itself link against to build the final Rice extension. Then you need to be able to link against the Ruby library to complete the process.

That said I'm not an expert at library compilation capabilities so it may be possible to have a build of Rice that does all compilation in one shot and get it working under static Ruby. I put that warning text in place a long time ago and don't exactly remember what I was running into at the time.

[Kagetsuki]

Aaah. Well you're awesome too @schneems . If you get any info please give us a heads up. I still owe you for helping me out before - you continue to rock.

[Kagetsuki]

@jasonroelofs I've spent way too much of my life messing with gcc/binutils to not give this a shot. I'm guessing if I wanted to try some things I should be looking at ruby/lib/mkmf-rice.rb.in first? Any pointers on what to start poking and prodding?

I'm guessing if I did get static compilation to work the biggest downside would be size since you're basically going to be smashing libruby into everything. That's an extra 13MB and change for every native.

[jasonroelofs]

@Kagetsuki Yeah that sounds about right. The extconf.rb builds up the librice.a file on gem install, then the require "mkmf-rice" sets up compilation such that your extension can include Rice headers and automatically link against librice.a and libruby.so.

[Kagetsuki]

@jasonroelofs Thanks, I'll give it a shot!

[Kagetsuki]

Quick question: IF we can link static, SHOULD we link static by default? I ask because when you release precompiled gems on rubygems you'd need to have them static to be compatible with rubies not compiled with dynamic support (like Heroku and I think defaults for rbenv). At the same time if you had a dynamic ruby it'd end up being more bulk and I guess there is also the possibility there could be some issues if you have something else running a dynamic link to a slightly different libruby.

[jasonroelofs]

@Kagetsuki That makes sense, and providing a binary gem would be nice, as it does take quite a long time to gem install rice due to the precompliation requirement.

[Kagetsuki]

@jasonroelofs Just for clarification I'm talking about gems complied with rice, not rice itself. I'm pretty sure providing a precompiled version of rice could amplify problems if there were any issues like libruby version incompatibilities etc. I mentioned above. I'm not super sure any of these issues would actually arise but seeing as to how rubygems/bundler have such rough granularity on specifying binary versions I'm not liking the idea. I was more talking about the actual compilation done by rice, EG gems like my rapngasm.

[Kagetsuki]

So I played around a bit and ended up directly modifying the Makefile to see if I could get a good combination. The key lines here ended up being:

RUBY_CFLAGS =  -O3 -fno-fast-math -static -pthread -fPIE -fPIC -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wunused-variable -Wpointer-arith -Wwrite-strings -Wdeprecated-declarations -Wno-packed-bitfield-compat
RUBY_CPPFLAGS = -I/home/zero/.rvm/rubies/ruby-2.2.3/include/ruby-2.2.0 -I/home/zero/.rvm/rubies/ruby-2.2.3/include/ruby-2.2.0/x86_64-linux -I/home/zero/.rvm/rubies/ruby-2.2.3/include/ruby-2.2.0/x86_64-linux
RUBY_CXXFLAGS =  -O3 -fno-fast-math -pthread -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wunused-variable -Wpointer-arith -Wwrite-strings -Wdeprecated-declarations -Wno-packed-bitfield-compat
RUBY_LDFLAGS = -L/home/zero/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/x86_64-linux -L/home/zero/.rvm/rubies/ruby-2.2.3/lib -L. -fstack-protector
RUBY_LIBRUBYARG = -Wl,-R/home/zero/.rvm/rubies/ruby-2.2.3/lib -L/home/zero/.rvm/rubies/ruby-2.2.3/lib -lruby
RUBY_LIBRUBYARG_STATIC = -fPIC -Wl,-R/home/zero/.rvm/rubies/ruby-2.2.3/lib -L/home/zero/.rvm/rubies/ruby-2.2.3/lib -pie
RUBY_LIBS = -lruby-static -lpthread -ldl -lcrypt

which gives me 100% passing on unittest AND to confirm with ldd:

        linux-vdso.so.1 =>  (0x00007ffd9e9bf000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f41ca9e9000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f41ca7e5000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f41ca5ad000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f41ca29e000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f41c9f96000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f41c9d80000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f41c99b6000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f41cb1ef000)

BAM! No libruby! A non static build ldd for comparison:

        linux-vdso.so.1 =>  (0x00007fff46f1e000)
        libruby.so.2.2 => /home/zero/.rvm/rubies/ruby-2.2.3/lib/libruby.so.2.2 (0x00007f2171fbb000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f2171cac000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f21718e2000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f21716cc000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f21714ae000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f21712aa000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f2171072000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f2170d6a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f217249a000)

But.... Don't get excited yet. vm_unittest gives me:

VM:create_object /home/zero/.rvm/rubies/ruby-2.2.3/lib/ruby/site_ruby/2.2.0/rubygems.rb:9:in `require': incompatible library version - /home/zero/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/x86_64-linux/thread.so (LoadError)
        from /home/zero/.rvm/rubies/ruby-2.2.3/lib/ruby/site_ruby/2.2.0/rubygems.rb:9:in `<top (required)>'
        from <internal:gem_prelude>:1:in `require'
        from <internal:gem_prelude>:1:in `<compiled>'
.

1 test(s), 0 assertion(s), 0 error(s), 0 failure(s)

Was this the issue you encountered? Any ideas?

[jasonroelofs]

@Kagetsuki About the binary builds: right, that makes sense.

And yes, regarding the vm_unittest errors, this suddenly makes a lot of sense. The details I ran into are hazy but I believe they were more to do with the static global variables that Rice::VM defines for older versions of Ruby. It's possible this is no longer relevant and the vm_unittest binary just needs to change how its built.

I have to be honest I have no idea who or if anyone actually uses the VM side of things, and maybe this is something that could be pulled out into a separate piece?

[Kagetsuki]

@jasonroelofs I'm going to be straight an honest here: I have no idea what the "VM side of things" even means so I'll leave such decisions up to you.

Would you like me to do anything to proceed?

[jasonroelofs]

Sorry, the Rice::VM class exists to lets you start up a Ruby interpreter in your C++ application and expose Ruby that way. it's weird to think of having a Ruby extension which itself starts up a Ruby interpreter ... and conflicts with all sorts of global state MRI keeps track of.

I need to carve out some time to go through this and a few other Issues and give some thought to where this should go. If leaving the VM class out of Rice means we can work with static-build Ruby, that would be a worthwhile trade-off IMO.

Thanks for your help looking into this!

[Kagetsuki]

@jasonroelofs That sounds like an entirely different type of project. I vote breaking this VM out if there's no need of it to perform the core rice task of building Ruby interfaces to C++.

Give me a heads up if there is something I can help with. I'm not good for time this month (and actually quite behind after blowing half the week messing with native builds on Heroku) but I'd love to see this move along. If necessary I think I can loose a few more hours of sleep here and there :grin:

[jasonroelofs]

Ha! I'll let you know if I run into something that you can help with, but I'd hate to be responsible for someone losing sleep!

[Kagetsuki]

@jasonroelofs I've got a 9 month old and two kids in elementary school, I run a small startup still in the red with a product [emojidex] that always needs more work done and I'm a member of an armature motor-sports team. You wouldn't be the first reason for me to loose sleep by a long-shot.

[jasonroelofs]

@Kagetsuki I've got a branch up that clears out Rice::VM and lets static builds run through. Everything seems to work fine on OS X and Ubuntu (15.10). Let me know if you get a chance to test this build out. I'll be doing my own beating on things and finally taking care of a few other Issues that have been open for a while!

[Kagetsuki]

Rock on @jasonroelofs !

I'm just finishing up some tasks right now but I'll give it a test on Ubuntu 15.04, 14.04, Debian and for the hell of it I'll give it a shot on my Windows VM because I actually had to test a build of something else on it anyway.

Edit: I'll test with a system ruby on 14.04, 15.04 and Debian I'm using RVM.

[Kagetsuki]

I can confirm it still works great with dynamic Ruby on all the platforms I mentioned above other than Windows (not tested yet). I'm trying to compile a static Ruby to test now but hitting weird issues with libssl linking. I'll comment directly on the PR once I have static ruby tested.

[Kagetsuki]

I hate to say it but I can't get ruby to build statically locally. I figured I may as well test on Heroku Cedar 14 anyway so I did. I get the following error during build:

Making all in ext
make[2]: Entering directory `/app/rice/test/ext'
/app/vendor/ruby-2.2.0/bin/ruby -I/app/rice/ruby/lib -C t1 extconf.rb --with-cppflags="-I/app/rice" --with-libpath="/app/rice/rice"
creating Makefile
/app/vendor/ruby-2.2.0/bin/ruby -I/app/rice/ruby/lib -C t2 extconf.rb --with-cppflags="-I/app/rice" --with-libpath="/app/rice/rice"
creating Makefile
make -C t1 all
make[3]: Entering directory `/app/rice/test/ext/t1'
compiling t1.cpp
cc1plus: warning: command line option ‘-Wdeclaration-after-statement’ is valid for C/ObjC but not for C++ [enabled by default]
cc1plus: warning: command line option ‘-Wimplicit-function-declaration’ is valid for C/ObjC but not for C++ [enabled by default]
linking shared-object t1.so
/usr/bin/ld: /app/vendor/ruby-2.2.0/lib/libruby-static.a(encoding.o): relocation R_X86_64_PC32 against symbol `rb_enc_find_index' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make[3]: *** [t1.so] Error 1
make[3]: Leaving directory `/app/rice/test/ext/t1'
make[2]: *** [t1_all] Error 2
make[2]: Leaving directory `/app/rice/test/ext'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/app/rice/test'
make: *** [all-recursive] Error 1

And I'm once again reminded why I generally dislike dealing with anything static.

@schneems perhaps you can have a test build and maybe give us a hint?

I'm firing up a fresh Debian VM and I'm going to try a static build without RVM. If I get a different result I'll post here.

[jasonroelofs]

@Kagetsuki And on Mac RVM builds static Ruby by default! A strange world we live in.

That linker error looks like we're just missing a flag when building some pieces of the tests, shouldn't be hard to fix.

[Kagetsuki]

@jasonroelofs I didn't even know that about RVM on OSX.

Thing about the flags is it looked like -fPIC was included. Ping me if you make a commit and I'll test it right away. In the mean time I'm still futzing with static builds on Ubuntu and I think I broke my default OpenSSL libs...

[schneems]

I'll preface this by saying I know very little about static binary building minutae, but I do know about Heroku.

Was that example running on deploy?

make[2]: Entering directory `/app/rice/test/ext'
/app/vendor/ruby-2.2.0/bin/ruby

When you deploy you won't be in /app you'll be in another directory that is passed into the buildpack https://devcenter.heroku.com/articles/buildpack-api#bin-compile that might be a factor.

[Kagetsuki]

@schneems Actually I just attached a bash session on a vanilla Cedar 14 instance and ran the build process by hand. Ghetto style :facepunch:

[schneems]

/s/Ghetto style/Hacker style/ :smile:

[Kagetsuki]

@jasonroelofs any updates here?

Just wanted to mention I tired using GH rice here directly on a Heroku build and it choked with the same error, but it looks like by specifying rice with the GH repo in the gemfile triggers a full build of rice including the tests and that's where it gets caught up.

Tell me if you want me to test or try anything.

[jasonroelofs]

I have not made any progress on this problem explicitly. I should be able to spend more time this weekend to see what I can track down.

[Kagetsuki]

@jasonroelofs OK, not trying to put any pressure on you though. Just tell me if there's anything I can do to help.

[jasonroelofs]

As far as I can tell this is going back to building-against-libruby-static that probably isn't fully solved yet. Also it looks like I'm wrong about RVM on OS X, I'm getting static and shared libraries built for any Ruby I install. Now that I'm trying to force actual static-only build for 2.2.3 under RVM, it's hitting failures trying to build the ext/-test-/ directories that I'm not sure how to fix. I have also run into the "incompatible library version" error with some other static linking tests I did.

It's starting to look like Ruby 2.x is not a big fan of static linking, and I'm having a hard time tracking down how C-extensions even work on Heroku with a static-built Ruby. If you've got any insight or ideas @Kagetsuki I'd love to hear them. Btw, compiling static builds on rvm is rvm --static install [version].

[Kagetsuki]

@jasonroelofs On both my desktop and notebook I got libssl linking errors when I tried to install a static ruby with RVM (using the command you mention).

Heroku basically has a super minimalist environment with a static ruby. Basically there is this stripped Ubuntu 14.04 image and on top of that a build-pack sets up a static Ruby of the version specified (usually in your Gemfile) on deployment. The problem here is that we have little solid information about how that Ruby is specifically composed. I can confirm there is a libruby.a, but anything over 2.2.0 seems to be unworkable (incompatible library errors).

@schneems , just now when I wrote the above sentence I realized something. libruby for 2.2.x is always versioned 2.2. This is going to sound stupid, but is it possible the 2.2.0 libruby, which has the same 2.2 version code as the libruby for 2.2.1, 2.2.2, and 2.2.3, is being used on every ruby deploy from 2.2.0 to 2.2.3? If so that would explain why we get no issues on 2.2.0 and issues on 2.2.1~2.2.3.

Otherwise there are several entries in http://svn.ruby-lang.org/repos/ruby/tags/v2_2_3/ChangeLog which catch my eye. EG:

Thu May 14 00:39:29 2015  Nobuyoshi Nakada  <nobu@ruby-lang.org>

    * dln.c (dln_load): check if a different libruby is loaded by the
      extension library, and then bail out to get rid of very frequent
      reported stale bug reports.

I still suspect something changed somewhere around 2.2.1 and it's messed everything up since

[hone]

@Kagetsuki if I understand what you're saying, no the libruby for 2.2.0 is not shared with 2.2.1/2.2.2/2.2.3/etc.

[Kagetsuki]

@hone Good, I should hope so! I hate to take up your time but can you give us some insight as to weather or not a dynamic Ruby would be possible on Heroku? Is there a specific reason that you know of all Rubies on Heroku are static?

[Kagetsuki]

Sorry to bump, but we've actually got an update for our site running on heroku and if we can't build we can't deploy. Anything I can do to help move this along?

EDIT: I'm grinding away at this now in a cedar-14 container. Hopefully I'll have a PR for you soon.

[Kagetsuki]

@hone @schneems So I tweaked some things and got a build working on the cedar-14 docker image. I grabbed an instance on heroku to double check. It's completely different. The ruby on an actual cedar-14 instance is absolutely different than the one on a cedar-14 container. Any ideas as to why this is or any suggestions as to how I can get the same ruby on a cedar-14 instance on a cedar-14 container?

[Kagetsuki]

@jasonroelofs I've spent the entire day banging away at this and I've made basically no progress. Part of the problem is I can't get a build of a static ruby that behaves the same way as the ruby on heroku. I can confirm though that the build flags @MikaelMollberg states in the top post are probably inaccurate (perhaps @hone or @schneems can comment on this?).

Can anyone give me a heads up if a build on a static ruby OTHER than the one on cedar-14 is giving the

relocation R_X86_64_PC32 against symbol `rb_enc_find_index' can not be used when making a shared object; recompile with -fPIC

error? If not I have a strong suspicion this is specifically an issue with how ruby on heroku is built and NOT an inherent issue with static rubies.

[jasonroelofs]

@Kagetsuki Sorry, I really don't have anything else to offer right now. I haven't been able to build a static Ruby on OS X that works and links with Rice. I get the same relocation errors that you are getting. I wish I had more to offer but I'm also not sure where to go from here.

[Kagetsuki]

@jasonroelofs

I get the same relocation errors that you are getting. I wish I had more to offer but I'm also not sure where to go from here.

Thanks for the confirmation!

What's confusing to me is that it appears everything is being built with -fPIC - can you confirm that everything in the chain should be being built with PIC?

It also occurs to me that PIC works differently in static and dynamic libraries. Maybe we should be building librice as dynamic? The thing is I'm still not completely sure how rice works so I'm not sure it would even be worth trying this.

If you can think of anything else give me a heads up and I'll give it a try.

[Kagetsuki]

I have not been able to get a build running on a static ruby BUT I did kludge a build and got it to run on a dyno using the following process:

1. Grab the heroku/ruby docker image
2. Remove ruby, recompile ruby 2.2.3 by hand (dynamic ruby)
3. Compile rice and my target gem
4. Copy libruby-static.a and dynamic ruby libs + compiled gems into the vendor folder of an app
5. Add vendor to the library load path, manually specify gem sources to unpacked binary gems
6. deploy

So, basically the static ruby on heroku WILL run dynamically linked things against a dynamic libruby - but it's an absolute mess to do and not anything maintanable. I don't find this to be an acceptable solution so I'm going to either keep trying to get static builds to work properly or see if I can get a dynamic ruby on heroku.

@schneems or @hone , I hate to keep bothering you, but can I please get a response as to why dynamic rubies are not available on heroku? I don't really understand how build packs work and I couldn't seem to find where the build args were in the ruby buildpack, so I'd really appreciate some information or some help.