neilhwatson / evolve_cfengine_freelib

Evolve Thinking's free Cfengine promise library.
18 stars 5 forks source link

new bundle for killing processes #18

Open neilhwatson opened 9 years ago

neilhwatson commented 9 years ago

A bundle to kill process that are long running, consuming too much memory, or not permitted by security. Consider not using CSV for this an all new bundles.

Parameters:

  1. Class
  2. Process command regex
  3. Process_owner regex list
  4. Minimum rsize in kilobytes
  5. Minimum elapsed process time in minutes
  6. Minimum number of processes

Because number of processes is not in a process select body, it will be tricky to combine. Will probably need three process promises.

basvandervlies commented 9 years ago

This is just an example. we use this to klll user processes that consume to much resources on our login nodes. This code must be run on compute nodes.

bundle agent sara_user_consume_resources(minutes, memory, exclude)
{
    processes:
        any::
            ".*"
                comment         => "Kill processes that use to much resources",
                signals         => { "term", "kill"},
                action          => sara_action,
                process_select  => sara_user_consume_resources_select("$(minutes)", "$(memory)", @(exclude));
}

##
# Kill processes that uses too much cpu time or memory and exclude
# important users
#
body process_select sara_user_consume_resources_select(minutes, memory, owners)
{
    command        => "ssh.*|scp.*|rsync.*|cp.*|mv.*|tar.*|gzip.*|bzip2.*";
    process_owner   => { @(owners) };
    ##
    # Bug in ps parsing minutes are seconds. so use the hour field for minutes
    # ttime_range     => irange(accumulated(0,0,0,"$(minutes)",0,0), accumulated(1,0,0,0,0,0));
    ttime_range     => irange(accumulated(0,0,0,0,"$(minutes)",0), accumulated(1,0,0,0,0,0));
    #
    rsize           => irange("$(memory)", 'inf');
    process_result  => "(ttime|rsize).!(process_owner|command)";
}
neilhwatson commented 9 years ago

@basvandervlies Is there a bug report for that time parsing problem?

basvandervlies commented 9 years ago

@neilhwatson this is fixed a long time ago. I must remove this statement. There is a bug report and pull request:

neilhwatson commented 9 years ago

@basvandervlies I think this bug is still true or the verbose output is off: snmp 2728 1 0 Aug22 ? 00:02:19 /usr/sbin/snmpd

verbose: P: Container path : '/default/main/processes/'/usr/sbin/snmpd.:'[0]' verbose: P: verbose: P: Comment: Set kill class if count too high verbose: P: ......................................................... verbose: verbose: Selection filter matched counter range 'TIME/TIME' = '00:02:19' in [60,999999999] (= 139 secs)

basvandervlies commented 9 years ago

Ok, i have the source for 3.6 and check it against the 3.7 version

On 24 aug. 2015, at 19:44, Neil H Watson notifications@github.com wrote:

@basvandervlies I think this bug is still true or the verbose output is off: snmp 2728 1 0 Aug22 ? 00:02:19 /usr/sbin/snmpd

verbose: P: Container path : '/default/main/processes/'/usr/sbin/snmpd.:'[0]' verbose: P: verbose: P: Comment: Set kill class if count too high verbose: P: ......................................................... verbose: verbose: Selection filter matched counter range 'TIME/TIME' = '00:02:19' in [60,999999999](= 139 secs)

— Reply to this email directly or view it on GitHub.


Bas van der Vlies | Operations, Support & Development | SURFsara | Science Park 140 | 1098 XG Amsterdam | T +31 (0) 20 800 1300 | bas.vandervlies@surfsara.nl | www.surfsara.nl |

neilhwatson commented 8 years ago

https://dev.cfengine.com/issues/7516

neilhwatson commented 8 years ago

new bug https://tracker.mender.io/browse/CFE-2115?jql="External%20issue%20id"~"7516"