search

ar- / incron

cron-like daemon which handles filesystem events
Other
229 stars 51 forks source link

Incron 0.5.12 not properly stopping forks on CentOS causing huge performance issues #53

Closed jadbaz closed 5 years ago

jadbaz commented 5 years ago

I've tried this on CentOS 7.5, 7.6 and 6.10 for incron 0.5.12 and 0.5.10 It seems like incrond 0.5.12 is not properly stopping its forks and resulting in huge performance issues.

Lingering processes

I'm creating a simple file watcher for a directory that runs true on IN_CREATE. That's it. My test creates 10 files and checks running processes.

incrond 0.5.10

Config

yum -y remove incron && http://packages.psychotic.ninja/7/base/x86_64/RPMS//incron-0.5.10-9.el7.psychotic.x86_64.rpm
rm -rf /var/tmp/test && mkdir -p /var/tmp/test
echo "/var/tmp/test IN_CREATE true" > /etc/incron.d/test
systemctl restart incrond
sleep 1
for i in `seq 1 10`; do touch /var/tmp/test/$i; done
ps aux | grep incron | grep -v grep

Output

[root@localhost ~]# ps aux | grep incron | grep -v grep
root     15992  0.8  0.1  12776   816 ?        Ss   14:23   0:07 /usr/sbin/incrond

incrond 0.5.12

Config

yum -y remove incron && yum -y install http://packages.psychotic.ninja/7/base/x86_64/RPMS//incron-0.5.12-3.el7.psychotic.x86_64.rpm
rm -rf /var/tmp/test && mkdir -p /var/tmp/test
echo "/var/tmp/test IN_CREATE true" > /etc/incron.d/test
systemctl restart incrond
sleep 1
for i in `seq 1 10`; do touch /var/tmp/test/$i; done
ps aux | grep incron | grep -v grep

Output

[root@localhost ~]# ps aux | grep incron | grep -v grep
root     15149  0.0  0.2  12788  1008 ?        Ss   13:08   0:00 /usr/sbin/incrond
root     15160  0.0  0.1  12788   664 ?        S    13:16   0:00 /usr/sbin/incrond
root     15163  0.0  0.0  12788   224 ?        S    13:16   0:00 /usr/sbin/incrond
root     15166  0.0  0.0  12788   224 ?        S    13:16   0:00 /usr/sbin/incrond
root     15169  0.0  0.0  12788   224 ?        S    13:16   0:00 /usr/sbin/incrond
root     15171  0.0  0.0  12788   424 ?        S    13:16   0:00 /usr/sbin/incrond
root     15174  0.0  0.0  12788   424 ?        S    13:16   0:00 /usr/sbin/incrond
root     15178  0.0  0.0  12788   224 ?        S    13:16   0:00 /usr/sbin/incrond
root     15181  0.0  0.1  12788   668 ?        S    13:16   0:00 /usr/sbin/incrond
root     15184  0.0  0.0  12788   224 ?        S    13:16   0:00 /usr/sbin/incrond
root     15187  0.0  0.0  12788   224 ?        S    13:16   0:00 /usr/sbin/incrond

Performance

Test

We're running a simple test that creates 10,000 files with the above configuration. Run the following

mkdir -p /var/tmp/test; rm -rf /var/tmp/test/*; for i in `seq 1 10000`; do touch /var/tmp/test/$i; sleep 0.01; done

Run top

incrond 0.5.10

top - 14:30:07 up 9 days,  1:21,  2 users,  load average: 0.71, 0.57, 0.32
Tasks: 111 total,   1 running, 110 sleeping,   0 stopped,   0 zombie

incrond 0.5.12

(I had to stop this after 2,000)

top - 14:44:16 up 9 days,  1:35,  2 users,  load average: 132.00, 31.32, 10.51
Tasks: 1988 total,   1 running, 1987 sleeping,   0 stopped,   0 zombie
jadbaz commented 5 years ago

This seems to be discussed extensively in the issue right before this one: issues/52 So closing this as a duplicate.