RFE - Split out the pure Ruby version into its own library

[djberg96]

Currently using the pure Ruby backend still requires installation (and thus compilation) of the C extension. Setting ENV['NIO4R_PURE'] doesn't change that.

Consequently I would like to see the pure Ruby version split out into its own library. This would make installation easier (since e.g. it would be a separate entry in a Gemfile). I also think it would make overall maintenance easier as issues could be addressed separately.

Another option would be to add a pre-install hook that looks for ENV['NIO4R_PURE'] and removes the spec.extensions at install time.

The problem I'm facing in the wild is Docker on an M1 Mac, as it essentially hangs for at least an hour on my system while it tries to build nio4r. If I can use the pure Ruby version just for development/testing, that would be preferred.

[tarcieri]

FYI: the pure Ruby version is the worst implementation. I don’t think it should be split out (especially if it’s polluting the same namespace trying to provide the same API), although perhaps there could be some way to disable building the C extension

The problem I'm facing in the wild is Docker on an M1 Mac, as it essentially hangs for at least an hour on my system while it tries to build nio4r.

That sounds like an environmental issue. On my M1 Mac it’s roughly a minute inside of a Docker container

[djberg96]

That sounds like an environmental issue. On my M1 Mac it’s roughly a minute inside of a Docker container

Could be, but it's an environment I don't control. Maybe I can narrow it down for our devops folks, I assume there's some sort of compatibility magic happening between M1 and x86-64.

[tarcieri]

Are you using something like bind mounts or volumes with Docker? They're ludicrously slow on macOS

[djberg96]

Are you using something like bind mounts or volumes with Docker? They're ludicrously slow on macOS

Note that I see, so I think it's something to do with our image or the environment in general. That, or libjemalloc is the culprit.

I tried adding this to the Gemfile:

Gem.pre_install do |installer|
  if installer.spec.name == 'nio4r'
    installer.spec.extensions = nil
  end
end

That basically works, but bundler just gets stuck on something else later (not even an extension).

[djberg96]

@tarcieri Upon further review, I think this has something to do with libjemalloc and our (Ubuntu, Ruby 2.7) image. Are there any known issues with jemalloc and nio4r? Otherwise it's something on my/our end.

[tarcieri]

Nope, but if you're having trouble compiling code quickly, it almost certainly speaks to something with your Docker environment

[djberg96]

I tried Ubuntu & Debian images with jemalloc + a bundle install, and they worked fine, so it's something to do with our environment (and our images in particular).

Anyway, back to the original issue, I'll go ahead and close it out since you don't want to split it out and there's a workaround that end users can use.

Thanks!

[djberg96]

@tarcieri I know I'm OT here, but I think you simulate what I'm seeing locally like so:

# Dockerfile.ruby
FROM ruby:2.7 AS base

RUN apt-get update \
    && apt-get install --no-install-recommends -y \
    build-essential \
    curl \
    git \
    libpq-dev \
    libjemalloc-dev \
    libsqlite3-dev \
    libssl-dev \
    openssl

RUN rm -rf /var/lib/apt/lists/*
RUN rm -rf /usr/share/doc
RUN rm -rf /usr/share/man
RUN apt-get clean

RUN groupadd -g 1000 ruby
RUN useradd --create-home --no-log-init -u 1000 -g 1000 ruby

ENV LD_PRELOAD=$LD_PRELOAD:/usr/lib/x86_64-linux-gnu/libjemalloc.so

WORKDIR /app
RUN chown ruby:ruby -R /app

COPY --chown=ruby:ruby Gemfile .

USER ruby
RUN gem install bundler
RUN bundle install

CMD ["bash"]

The Gemfile is just:

source 'https://rubygems.org'
gem 'rails'
gem 'nio4r'

And the command to build: docker build -t ruby27 -f Dockerfile.ruby --platform x86_64 .