search

Copterfly / modwsgi

Automatically exported from code.google.com/p/modwsgi
0 stars 0 forks source link

Add limits on cpu time and maximum resident memory. #21

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
As an idea to controlling memory usage, provide a means of monitoring
memory usage and force a restart of the daemon processes when process
memory usage exceeds some set value.

Such a mechanism is provide with mod_perl:

  http://search.cpan.org/dist/mod_perl/lib/Apache/SizeLimit.pm
  http://books.google.com/books?q=set_min_shared_size&btnG=Search+Books

although it use is restricted to certain platforms. On Linux the smaps
information from the proc filesystem can be used. On true BSD systems (not
Mac OS X), getrusage() can be used.

Although the mod_perl implementation allows actions to be triggered off the
amount of shared memory dropping below a certain value, this is problematic
on some systems when a process is swapped. Thus concentrate just on total
process memory size, ie., unshared + shared. Although this counts shared, a
single value is probably easier to comprehend by shared web hosting setups,
who would be the primary target for this feature.

The name of the option for WSGIDaemonProcess might be 'maximum-memory'.

Note, would probably only look at implementing this for daemon processes
and not for main Apache child processes.

Original issue reported on code.google.com by Graham.Dumpleton@gmail.com on 10 Aug 2007 at 6:08

GoogleCodeExporter commented 9 years ago

Original comment by Graham.Dumpleton@gmail.com on 5 Sep 2007 at 11:41

GoogleCodeExporter commented 9 years ago 
Should probably just implement Apache RLimitCPU and RLimitMEM directive 
equivalents
for daemon processes. Options would be rlimit-cpu and rlimit-mem.

For CPU limit, signal indicating soft limit reached could cause orderly restart 
of
daemon process. This would help in blocking run away processes.

For MEM limit, no signal available, so all that would happen is that memory
allocations would start failing. Would need to make sure mod_wsgi at least 
handles
this gracefully in all cases. Currently may not check all returns form memory
allocating operations properly.

Original comment by Graham.Dumpleton@gmail.com on 26 Mar 2008 at 11:34

GoogleCodeExporter commented 9 years ago 
No need to implement equivalents to these Apache directives, the actual values 
associated with these Apache 
directives can actually be used by querying them out of core module config. 
This is how mod_cgi/mod_cgid 
work. The values should probably be applied in daemon process just prior to 
dropping root privileges.

Original comment by Graham.Dumpleton@gmail.com on 16 Apr 2008 at 11:02

GoogleCodeExporter commented 9 years ago 
If value of Apache directives used, can only grab it from either main server 
context or virtual host context. For 
CGI scripts it can also be defined in Directory or .htaccess context, but this 
doesn't make sense for mod_wsgi 
as daemon processes love longer than a single request. As such, memory limit 
may actually need to be much 
more than what one may define for a single request. The cpu limit would also 
likely need to be substantially 
more. Thus, specific options for WSGIDaemonProcess probably more appropriate 
after all.

Because the directive actually take one or two arguments, need to somehow allow 
that with 
WSGIDaemonProcess options. Because hard to have spaces in options for 
WSGIDaemonProcess, use a 
separator such as '/' instead. Thus might have:

  rlimit-cpu=120/150 rlimit-mem=134217728/max

Original comment by Graham.Dumpleton@gmail.com on 16 Apr 2008 at 11:16

GoogleCodeExporter commented 9 years ago

Original comment by Graham.Dumpleton@gmail.com on 17 Jul 2008 at 1:52

GoogleCodeExporter commented 9 years ago 
Been suggested that setrlimit() for RLIMIT_RSS doesn't work on Linux since 
2.4.30. See discussion:

  http://groups.google.com/group/modwsgi/browse_frm/thread/10aa3a607b40d67b?hl=en

This may make adding such a feature pointless.

Original comment by Graham.Dumpleton@gmail.com on 4 Sep 2008 at 11:14

GoogleCodeExporter commented 9 years ago 
Added cpu-time-limit option to WSGIDaemonProcess directive in revision 1224 of 
trunk for mod_wsgi 3.0.

Original comment by Graham.Dumpleton@gmail.com on 10 Mar 2009 at 10:57

GoogleCodeExporter commented 9 years ago

Original comment by Graham.Dumpleton@gmail.com on 10 Mar 2009 at 11:07

GoogleCodeExporter commented 9 years ago 
Because adding a limit on resident memory doesn't work at all on many Linux 
kernel versions, not bothering with 
implementing memory limits. Will create a separate issue for that in the future 
if way determined of enforcing 
limits.

Original comment by Graham.Dumpleton@gmail.com on 10 Mar 2009 at 11:08

GoogleCodeExporter commented 9 years ago 
Version 3.0 of mod_wsgi now released wuth cpu-time-limit option for 
WSGIDaemonProcess.

Original comment by Graham.Dumpleton@gmail.com on 22 Nov 2009 at 2:48