Child process erroring out instead of blocking on write to stdout on Ruby 3.0?

[ClearlyClaire]

I'm trying to port the application I am working on to Ruby 3.0, and one of the issues I have is some dependency “randomly” failing to run commands with large output. I have tracked down the issue to posix-spawn as removing that gem makes the library fall back to Process.spawn which seems to work as expected.

It seems that for some reason, when using posix-spawn on Ruby 3.0, the child process will error out with “Error 11: Resource temporarily unavailable” when filling the pipe's buffer, while on Ruby 2.7, it will simply block as expected.

The following simple code demonstrates the issue:

require 'posix/spawn'

pid, stdin, stdout, stderr = POSIX::Spawn.popen4('yes | head -n 20000000')
stdin.close
puts "output size: #{stdout.read.size}"
puts "stderr: #{stderr.read}"
stdout.close
stderr.close
Process.waitpid pid

[ClearlyClaire]

Writing a simple C program confirms that for some reason, stdout is set to non-blocking on Ruby 3.0, but not on Ruby 2.7:

#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>

int main(void) {
  printf("%d\n", fcntl(STDOUT_FILENO, F_GETFL) & O_NONBLOCK);

  return 0;
}

Indeed, unlike Ruby 2.7, Ruby 3.0 makes all pipes non-blocking by default, and as far as I understand it, manually undoes that when passing stdout etc. to child processes with built-in popen/exec etc.

This means that posix-spawn probably has to do the same thing, not only for POSIX::Spawn.popen4, but probably for POSIX::Spawn.spawn as well.

[catwell]

For reference this fork apparently solves the problem for spawn: https://github.com/jhawthorn/posix-spawn/commit/529e2147f6e900507efea2559562bd262d816ec3

@jhawthorn Is that patch used at GitHub? If so is there any chance to upstream it?

Regarding popen4 it can be patched directly into the posix-spawn Ruby code.

[jhawthorn]

@catwell we are no longer using this gem. Ruby has had a built-in and efficient Process.spawn for a long time so I don't think there's any reason to use this gem anymore.

[catwell]

OK, thanks for the answer! I was still using it in part because I prefer its API - see this blobpost by @jnunemaker for an example - but I can switch to native Process.spawn in this case.

I also suggest the maintainers of posix-spawn mark it as officially deprecated in the README.

[midnight-wonderer]

@catwell I was late to the party and could not quite follow.

Why did you suggest that the gem should be deprecated? What changed in the built-in Process.spawn? What do you suggest in place of POSIX::Spawn::Child?

[catwell]

@midnight-wonderer I only suggest the gem should be deprecated if the maintainers don't use it anymore and don't intend to fix its bugs.

Process.spawn is not what changed, it's IO (and pipes in particular) that is non-blocking by default. But pipes used for stdin / stderr / stdout should not be non-blocking, so they should be made blocking explicitly.

[midnight-wonderer]

Hey, thanks a bunch for the explanation; that answered most of my questions.

However,
from a couple of dialogs ago, it seems that GitHub didn't use the gem anymore and switched back to Process::spawn.
Made me wonder what's different from now and then when POSIX::Spawn was created; what was changed in Process::spawn?
How does it make a comeback?

I don't quite understand low-level details of OS kinds of stuff, and since you considered switching to Process::spawn too, so I asked in succession because you seem to understand that one too.

[catwell]

Not entirely sure that is what he was referring to but Ruby now uses vfork when it can. Like I said that wasn't my reason for using this gem anyway.

[midnight-wonderer]

That's it, thank you.
Basically, Ruby is using vfork natively now.

I just modified and ran a benchmark of POSIX::Spawn on my computer using ruby:3-slim Docker image on an Ubuntu 20.04 host. Here's the result:

$ bundle exec rake benchmark
cp tmp/x86_64-linux/posix_spawn_ext/3.1.2/posix_spawn_ext.so lib/posix_spawn_ext.so
/usr/local/bin/ruby -Ilib bin/posix-spawn-benchmark
benchmarking fork/exec vs. posix_spawn over 1000 runs at 100M res
                                               user     system      total        real
fspawn (fork/exec):                        0.097599   2.031621   6.810560 (  6.756999)
pspawn (posix_spawn):                      0.006890   0.025629   0.251916 (  0.244938)
spawn (native):                            0.003529   0.032049   0.195628 (  0.188914)
Child (posix_spawn):                       0.037435   0.037484   0.294511 (  0.275843)
open3 (native):                            0.058668   0.037705   0.262761 (  0.226236)

The open3 implementation is using similar IO::select as POSIX::Spawn::Child.
The posix-spawn gem is worse than native, time-wise.

The gem should not be used in new applications because it doesn't provide advantages over native Ruby implementation anymore. And GitHub + the gem maintainers have moved on.

For the Child API, I think it is trivial to implement something similar with Open3.

[catwell]

Yes, that is probably what I will do now. In the meantime we monkey patched this gem to have it work on Ruby 3 since we wanted to avoid such changes at the same time as our Ruby 3 upgrade.

[jjb]

does anyone know where I can find discussion about the changes with Process::spawn in ruby 3?

Searching the web, ruby codebase, and ruby changelogs for vfork doesn't come up with anything

• https://www.google.com/search?q=ruby+3+spawn
• https://www.google.com/search?q=ruby+3+vfork
• https://ml.ruby-lang.org/mailman3/hyperkitty/search?count=200&mlist=ruby-core%40ml.ruby-lang.org&q=vfork

This code was committed 9 years ago

https://github.com/ruby/ruby/blame/7b27ad9ad36dbfd2ec6571b0ed15fbc4aa65d460/process.c#L58

If anyone has better search terms for me to use, much appreciate, thanks!

[catwell]

@jjb Yes the vfork change is older than Ruby 3, it shipped with Ruby 2.2.

POSIX::Spawn is much older though, it was created for Ruby 1.9, back when fork performance was bad, as said at the top of the README:

So since Ruby 2.2 it is no longer necessary to use POSIX:Spawn to get good fork performance, but it still worked well until Ruby 3.

The breaking change in Ruby 3 is that pipes, used internally by POSIX:Spawn, became non-blocking by default, as explained earlier in the thread. If you're looking for the commit, that change happened here.

[jjb]

Thanks for that info! Wow, interesting - I didn't discover posix-spawn until far after 2.2 and used it everywhere I could.

I did understand that breaking change prompted the discussion in this ticket - what I was looking for was info about the change in ruby which made posix-spawn irrelevant, so I guess back in 2.2. Looks like the release notes cover it! https://www.ruby-lang.org/en/news/2014/12/25/ruby-2-2-0-released/

Experimental support for using vfork(2)....

https://github.com/ruby/ruby/commit/9b16f906922fcb63a35c41b57bcec91f5791ad66

and looks like it's never used on macos and solaris: https://github.com/ruby/ruby/blob/master/configure.ac