flavorjones commented 2 years ago

Since Bundler 2.3.21 / Rubygems 3.3.21, it's possible to ship a gem for the specific *-linux-musl platform.

For many gems, like Nokogiri, we've been able to figure out how to build binaries that work on both glibc and musl platforms, but other gems (like sass-ruby) had to stop precompiling because of incompatibilities; and it sounds like (based on a chat with @ianks) that the Rust toolchain handles musl systems very differently as well.

This is a placeholder to discuss if, and when, we should support x86_64-linux-musl and aarch64-linux-musl as distinct target platforms.

ianks commented 2 years ago

For additional context, here is the Dockerfile we use for x86_64-linux-musl. @eliias did all of the hard work on making it work properly, so looping in. From what I gather, it's mostly copy-pasta of what RCD already had... but could be wrong.

eliias commented 2 years ago

Hey folks,

as @ianks already wrote, the Dockerfile I provided is mostly a straightforward port of the RCD image, minus the manylinux specific setup parts, because I am relying on a base image that provides all the tools for musl (messense/rust-musl-cross:x86_64-musl, https://github.com/messense/rust-musl-cross/blob/main/Dockerfile) and this image is based on Ubuntu 22. The result of all that is that the image is quite large (it probably adds at least 10% to the image size).

For full disclosure, I once tried to build a basic image where I add the musl tool chain myself, but it took forever to build, and I was running into various issues, so the result is a shortcut. I was looking into https://musl.cc/, and there are indeed some pre-compiled binaries we could use, it is probably just some extra work to add them.

As a side note, my primary motivation was to support builds for Alpine.

flavorjones commented 1 year ago

Here's an interesting failure case for how we're currently handling musl (which is, basically, hoping and testing):

https://github.com/sparklemotion/sqlite3-ruby/issues/372

Some prior art I was able to find around generating musl binaries:

ianks commented 1 year ago

@flavorjones Have you given a shot just directly using RCD_IMAGE=rbsys/x86_64-linux-musl:latest? Curious if it would Just Work™️

CAMOBAP commented 1 year ago

@flavorjones Have you given a shot just directly using RCD_IMAGE=rbsys/x86_64-linux-musl:latest? Curious if it would Just Work™️

@ianks thanks a lot for suggesting image (it works in my case), BTW maybe you can suggest a good candidate for aarch64-linux-musl?

ianks commented 1 year ago

@flavorjones Have you given a shot just directly using RCD_IMAGE=rbsys/x86_64-linux-musl:latest? Curious if it would Just Work™️

@ianks thanks a lot for suggesting image (it works in my case), BTW maybe you can suggest a good candidate for aarch64-linux-musl?

As of now, there is no supported rake-compiler-dock image for aarch64-linux-musl unfortunately.

segiddins commented 1 year ago

What would be needed to make this happen? I'm currently looking into revamping rubygems' support for "platforms", and as a part of that we want to make shipping precompiled extensions as ubiquitous as possible, so I'd love to understand what blocks cross-compiling against musl here

flavorjones commented 1 year ago

we want to make shipping precompiled extensions as ubiquitous as possible

@segiddins This is very very interesting to me (I've given multiple conference talks (1, 2) on precompiled gems; and help maintain precompiled gems nokogiri, sqlite3, and ruby-magic, as well as this project; and created a project explaining precompilation with working examples and CI) ... and probably to @larskanis and others as well. How can we help support this effort?

This issue is to discuss a (slowly) ongoing project to create a docker image suitable for cross-compiling for musl. This is pretty much just a time-and-effort question, since @eliias and @ianks have already done something specifically for rust extensions (see Ian's comment above). I explored using the rb-sys image with nokogiri but ran into some errors that I didn't have time to finish investigating.

I'll try to carve out some time this weekend to rebuild context and document where support stands.

flavorjones commented 1 year ago

Closing the loop, I did find time to explore this again over the weekend, and although I can precompile a working nokogiri gem for MUSL, it's not reliable and I think this is because rake-compiler doesn't know how to handle the difference between x86_64-linux and x86_64-linux-musl.

I've timed out for today, but I'm pretty confident I can figure this out with a bit more work.

ianks commented 1 year ago

@segiddins This effort would be very much appreciated. If you want to chat or discuss ideas, please don’t hesitate to contact me. Very exciting.

eliias commented 1 year ago

Closing the loop, I did find time to explore this again over the weekend, and although I can precompile a working nokogiri gem for MUSL, it's not reliable and I think this is because rake-compiler doesn't know how to handle the difference between x86_64-linux and x86_64-linux-musl.

I've timed out for today, but I'm pretty confident I can figure this out with a bit more work.

@flavorjones IIRC I added this line to make it work, pretty hacky but doing the job https://github.com/oxidize-rb/rb-sys/blob/main/docker/Dockerfile.x86_64-linux-musl#L127

flavorjones commented 11 months ago

Picking this back up again today due to the recent musl changes in alpine:3.19.