search

stripe / subprocess

A port of Python's subprocess module to Ruby
MIT License
208 stars 17 forks source link

On spawning multiple processes, the processes are going into sleep state. #65

Open akhileshnirna opened 3 years ago

akhileshnirna commented 3 years ago

Hello, I've been conducting load tests on my code on AWS EC2 instance. This piece of code is run on multiple threads. (it executes a ruby script written by me):

begin
  Subprocess.check_call(['some','command'])
rescue Subprocess::NonZeroExit => e
  p e.message
end

OS: Ubuntu No of threads: 15 AWS Instance: t2.2xlarge

After some amount of time, i start noticing that most of the processes go into S state and stay there Following is the output of top command:

Screen Shot 2021-02-10 at 5 18 16 PM

Note: I have ran my tests on the same code without using subprocess (i.e. only multithreading), it works fine without any issues. However there is a requirement that I use Subprocesses.

Could you please guide as to where I am going wrong. Is it possible that the pipe buffer is being filled? I am not using pipe buffers for my std-in/out/err and the default behavior should not be using pipe.

Thanks, Akhilesh