No limit on number of processes

[danfruehauf]

When executing commands for new files that arrive, there is no limit to the number of fork() calls.

Imagine the following scenario:

• A thousand files land up more or less at the same time on a watched directory

What happens:

• A thousand fork() calls will be called, resulting in a thousand new processes, which can easily jam the system

What should happen:

• incron should have an upper limit on the number of active processes it starts
• Can potentially be implemented with a queue and a thread/process pool

[demestav]

I also believe that his feature is important. If the rate at which files are arriving to the directory is higher than the rate they are processed then soon enough there will be trouble!

[danfruehauf]

If the rate at which files are arriving to the directory is higher than the rate they are processed then soon enough there will be trouble!

@demestav Why not prevent the trouble then?

[demestav]

@danfruehauf In order to prevent the trouble, I would need incrond to limit the number of processes so the files are processes in a controlled way. For example, every time at some specific tasks, I have tens of files arriving into a specific directory for processing. So, every 5 seconds a new file arrives in the directory but it takes about 20 seconds for that file to get processed. Soon enough the number of processes spawned by incrond overwhelms the system. If there was an upper limit to the number of processes, the all the files would be processed in a controlled way, without overwhelming the system.

[danfruehauf]

@demestav Sorry about my previous comment, I misread you. We simply agree on that matter.