rake-compiler / rake-compiler-dock

Easy to use and reliable cross compiler environment for building Windows, Linux, Mac and JRuby binary gems.
MIT License
78 stars 30 forks source link

Release with Ruby 2.7 support? #30

Closed apolcyn closed 4 years ago

apolcyn commented 4 years ago

For the "grpc" gem, we currently use a rake-compiler-dock docker image as a base for a slightly customized dockerfile, see https://github.com/grpc/grpc/blob/c4f80b8c877e3ecf259c4da3e8f6be06e1ff8450/third_party/rake-compiler-dock/Dockerfile

To add ruby 2.7 support to grpc (https://github.com/grpc/grpc/issues/21514), it would be convenient to use a published rake-compiler-dock image which has add ruby 2.7 cross compiling support.

Just wondering plans/timelines for the next rake-compiler-dock release with ruby 2.7 support

methodmissing commented 4 years ago

Interested too - been following along the recent changes in both projects and https://github.com/ruby/ruby/pull/2772

larskanis commented 4 years ago

@apolcyn I'm about to release rake-compiler-dock-1.0.0. It adds ruby-2.7, but the big change is the use of dedicated docker images per cross build target. So there are 4 MRI images currently and one for JRuby. They share common docker layers where possible to avoid increased download sizes. I'll document that in the CHANGELOG.

I would like to incorporate the changes of your Dockerfile into rake-compiler-dock since they are possibly useful for others.

Line 8 seems to be superfluous since this is already installed in the base image.

The switch to pthread could be interesting for other projects as well. Is there any drawback in using them?

I'm uncertain, what the find lines really are for. Do you know?

I didn't find a description of how to build grpc fat binary gems locally and didn't try anything so far. Is it difficult?

apolcyn commented 4 years ago

cc @nicolasnoble and @veblush

I didn't find a description of how to build grpc fat binary gems locally and didn't try anything so far. Is it difficult?

@larskanis the grpc binary packages can be built locally with the following steps:

$ git clone https://github.com/grpc/grpc.git --recursive
$ cd grpc
$ bundle install

# This next command will take a while, removing ruby versions and platforms
# from the cross-compilation matrix can speed it up. Note that this rake task
# internally invokes RakeCompilerDock.sh with the "rake cross native gem ... " command
$ bundle exec rake gem:native
$ cd pkg # binary packages are now in here
nicolasnoble commented 4 years ago

The various find commands do some various alterations to the compilation environment. Are all of them confusing or only certain ones?

methodmissing commented 4 years ago

@larskanis I've been using rake-compiler for many years and rake-compiler-dock for almost a year and let it be known that X compiling and fat gem distributions is a HARD problem. :clap: for packaging it into an easy to use solution AND maintaining it very well

larskanis commented 4 years ago

I added all changes from the grpc Dockerfile to rake-compiler-dock-1.0.0 except the rename of gettimeofday(). I guess there's some conflict with this method, but I don't want to change the public API of Ruby. Moreover this function is used by the pg.gem. I think there's probably another way to fix this issue for grpc or to apply the change in the RakeCompilerDock.sh call.

Another questionable change for me is the rename of libruby.dll.a to just libruby.a . The linker should equally link to the dll.a version, but since this change doesn't hurt, I merged it. Can you describe the rationale behind it?

As discussed above, rake-compiler-dock-1.0.0 has some changes to the rake tasks that are described in the README. You can have a look into pg gem or fxruby or nokogiri, how these changes look, if you're intend to use the parallel build feature.

veblush commented 4 years ago

There is no feasible way to use C++ thread-related standard library such as mutex on mingw except using pthreads-win32. It may introduce a bit of overhead due to a wrapping layer between this but it only happens when using those libraries, I believe.

larskanis commented 4 years ago

OK, support for ruby-2.7 is out since 3 months and grpc.gem is running well with rake-compiler-dock-1.0 (as far as I know). I'll check at future releases that grpc still works. Closing this issue now.