posix_spawn and jruby

[gustavobap]

Hello, I'm using the gem paperclip with jruby, I have a memory problem due to the way terrapin make OS calls when running in a java environment. It does not support ProccessRunner or PosixRunner, so when the call is made in the standard way (backticks) the rails process is forked in the normal way. The problem is when the fork occur the stack from the parent process has to be copied to the child, what is a problem in this environment because jruby is too memory hungry.

The PosixRunner is supported when the gem posix-spawn is installed, but it is not compatible with jruby. Therefore, I'm trying do adapt the PosixRunner to create the SpoonRunner, which use the gem spoon that is compatible with jruby and does the same thing as posix-spawn. Is this a good approach ? Someone has done this ?

The problem is I can't figure out how to work with file descriptors in order to get the output value of the command being executed. I would really appreciate some help with this = )

Please check the comments on the code bellow:

    module Terrapin
      class CommandLine
        class SpoonRunner
          def self.available?
            return @available unless @available.nil?

            @available = spoon_gem_available?
          end

          def self.supported?
            available?
          end

          def supported?
            self.class.supported?
          end

          def call(command, env = {}, options = {})
            pipe = MultiPipe.new
            pid = spawn(env, command, options.merge(pipe.pipe_options))
            pipe.read_and_then do
              waitpid(pid)
            end
            pipe.output
          end

          private

          def spawn(env, command, options)

            # spoon gem example
            # file_actions = Spoon::FileActions.new
            # file_actions.close(1)
            # the first argument is file_descriptor
            # file_actions.open(1, "/tmp/ls.out", File::WRONLY | File::TRUNC | File::CREAT, 0600)
            # spawn_attr = Spoon::SpawnAttributes.new
            # pid = Spoon.posix_spawn('/usr/bin/env', file_actions, spawn_attr, %w(env ls -R))

            puts "OPTIONS: #{options.inspect}"
            # OPTIONS: #<IO:fd 27>

            # why do I have to specify the file descriptor and the path, if the fd already exists
            # does it mean it is already associated to a path ? Is there a way I can use a random
            # one here and also an unique file name in order to keep things thread safe ?
            # I guess maybe I have to use the fd passed in options here and write on this file
            # that already exists, how can I do that ?

            fd = ?
            path = ?
            flags = ?

            file_actions = Spoon::FileActions.new
            # do I have to close it first ?
            file_actions.close(fd)
            file_actions.open(fd, path, flags, 0600)
            spawn_attr = Spoon::SpawnAttributes.new

            pid = Spoon.posix_spawn(hash_to_path_env(env), file_actions, spawn_attr, [command])
          end

          def hash_to_path_env(hash)
            # TODO
            return ""
          end

          def waitpid(pid)
            Process.waitpid(pid)
          end

          def self.spoon_gem_available?
            require 'spoon'
            true
          rescue LoadError
            false
          end

          private_class_method :spoon_gem_available?
        end
      end
    end

Thank you !

[Unknown-Guy]

Have app with lots of legacy still on Centos7, this is how I did this with spoon

require 'shellwords'

module Terrapin
  class CommandLine
    class SpoonRunner
      def self.available?
        return @available unless @available.nil?

        @available = spoon_gem_available?
      end

      def self.supported?
        available? && OS.java?
      end

      def supported?
        self.class.supported?
      end

      def call(command, env = {}, options = {})
        pipe = MultiPipe.new

        fileactions = Spoon::FileActions.new
        fileactions.close(0)
        fileactions.dup2(pipe.pipe_options[:out].fileno, 1)
        fileactions.dup2(pipe.pipe_options[:err].fileno, 2)

        spawn_attr = Spoon::SpawnAttributes.new
        argv = to_argv(command)

        pid = Spoon.posix_spawnp(argv[0], fileactions, spawn_attr, argv)

        pipe.read_and_then do
          Process.waitpid(pid)
        end

        pipe.output
      end

      private

      def to_argv(command)
         Shellwords.split(command)
      end

      def self.spoon_gem_available?
        require 'spoon'
        true
      rescue LoadError
        false
      end

      private_class_method :spoon_gem_available?
    end
  end
end