Pulled Pork goes in infinite loop

[GoogleCodeExporter]

> ps ax |grep pulled

root 1885  0.0  0.0  11620 328  ? Ss Nov09     0:00 /bin/sh -c 
/etc/pulledpork/pulledpork_update.sh >> /var/log/snort/pulledpork.log 2>&1
root 1886  0.0  0.0  11624 372  ? S  Nov09     0:00 /bin/bash 
/etc/pulledpork/pulledpork_update.sh
root 1917 99.3  0.2 113528 5464 ? R  Nov09 40665:44 /usr/bin/perl 
/etc/pulledpork/pulledpork.pl -c /etc/pulledpork/pulledpork.conf

Notice the last line. That process has been running from "Nov 9" and has taken 
up lots of CPU seconds.

This same thing I have noticed 3-4 times in other machines as well.

It appears that somewhere pulledpork is going in infinite loop.

What steps will reproduce the problem?
1. Do not know how to reproduce but it has happened 3-4 times already.
2. It does not happen always but once in a while keeps happening.

What is the expected output? What do you see instead?
Pulled pork should have finished (or timedout) long back.

What version of the product are you using? On what operating system?
Pulled pork 6.1
Fedora 16 64 bit.
Kernel 3.3.0-8
Perl 5.14.3

Please provide any additional information below.

I am using emergingthreats rules.

I have slightly modified it (just regex) based on online research, so that it 
works for emergeingthreats and https.

Attached is diff file. To indicate what code I have modified.

Also attached my conf file. Not much changes. Just paths. "distro=FreeBSD-8.0" 
line I have commented.

disabledsid.conf has these:
pcre:P2P.+Connection
pcre:P2P.+Traffic

dropsid.conf has these:
ET-emerging-attack_response
ET-emerging-botcc
ET-emerging-exploit
ET-emerging-virus
ET-emerging-current_events

enablesid.conf and modifysid.conf

So please help me debug and resolve this issue. As otherwise pulledpork takes 
away lots of CPU power.

Thank you.

Original issue reported on code.google.com by amis...@gmail.com on 7 Dec 2012 at 5:04

Attachments:

• pp.diff
• pulledpork.conf

[GoogleCodeExporter]

I would like to see the output from -vv when it goes in an infinite loop, any 
way that you can get this?

Original comment by Cummin...@gmail.com on 11 Dec 2012 at 3:33

[GoogleCodeExporter]

Well its a process that is run by cron. I can not do -vv on production 
environment.

Reason being that pulledpork with -vv if goes in loop can fill up harddisk 
easily (overnight when i am asleep and not monitoring in daytime)

This issue appears under rare case. I have not been able to re-produce this 
loop when running pulled pork manually.

Now I have added ulimit -t 60 in script that calls pulledpork. So it kills 
pulledpork automatically if it takes so much CPU.

Minimal logging is still on. So I am now waiting for it to happen again.

Earlier log was lost because of log rotation every 5 days.

Original comment by amis...@gmail.com on 11 Dec 2012 at 3:58

[GoogleCodeExporter]

Ok I think I have found the issue. Not 100% sure but this seems highly likely 
problem.

compare_md5() is a recursive function.

If for some reason rulefetch() fails or md5sum() fails OR is different from 
rule file that is fetched, then this function will simply keep on calling 
itself over and over again infinitely.

May be we need some sort of break away counter there. Say function should fail 
completely if md5sum still dont match after 3 or 4 tries.

So please patch if possible.

Thank you.

Original comment by amis...@gmail.com on 11 Dec 2012 at 4:17

[GoogleCodeExporter]

Ok this issue occurred two days in a row. It looks like that this happens when 
rule file is partially downloaded.

Here is output with -vv added in shell script. I dont know if much can be 
deduced from the output but here it is:

Running PulledPork.
Config File Variable Debug /etc/pulledpork/pulledpork.conf
        snort_path = /usr/sbin/snort
        enablesid = /etc/pulledpork/enablesid.conf
        temp_path = /tmp
        version = 0.6.0
        modifysid = /etc/pulledpork/modifysid.conf
        disablesid = /etc/pulledpork/disablesid.conf
        rule_path = /etc/snort/rules/snort.rules
        ignore = deleted,experimental,local,emerging-botcc-BLOCK,emerging-compromised-BLOCK,emerging-drop-BLOCK,emerging-dshield-BLOCK,emerging-rbn-BLOCK,emerging-rbn-malvertisers-BLOCK,emerging-tor-BLOCK
        dropsid = /etc/pulledpork/dropsid.conf
        rule_url = ARRAY(0x17d16d8)
        sid_changelog = /var/log/snort/sid_changes.log
        sid_msg = /etc/snort/sid-msg.map
        local_rules = /etc/snort/rules/local.rules
        config_path = /etc/snort/snort.conf
** GET 
https://rules.emergingthreatspro.com/open/snort-2.9.4/emerging.rules.tar.gz.md5 
==> 100%^H^H^H^H200 OK (3s)
** GET 
https://rules.emergingthreatspro.com/open/snort-2.9.4/emerging.rules.tar.gz ==> 
  0%^H^H^H^H  1%^H^H^H^H  2%^H^H^H^H  3%^H^H^H^H  4%^H^H^H^H  5%^H^H^H^H  
6%^H^H^H^H  7%^H^H^H^H  8%^H^H^H^H  9%^H^H^H^H 10%^H^H^H^H 11%^H^H^H^H 
12%^H^H^H^H 13%^H^H^H^H 14%^H^H^H^H 15%^H^H^H^H 16%^H^H^H^H 17%^H^H^H^H 
18%^H^H^H^H 19%^H^H^H^H 20%^H^H^H^H 21%^H^H^H^H 22%^H^H^H^H 23%^H^H^H^H 
24%^H^H^H^H 25%^H^H^H^H
/etc/pulledpork/pulledpork_update.sh: line 40: 32592 Killed 
/etc/pulledpork/pulledpork.pl -vv -c /etc/pulledpork/pulledpork.conf

The pulledpork is killed because I have added ulimit -t 60 in script.

I doubt that pulledpork in normal cases will take 60 seconds of CPU time.

The file /tmp/emerging.rules.tar.gz is downloaded partially.(just around 300KB 
instead of what should be more than 1MB)

On checking md5sum:

> md5sum /tmp/emerging.rules.tar.gz
e986ed021a2a63dde1517b4d2df655f8  /tmp/emerging.rules.tar.gz
> cat /tmp/emerging.rules.tar.gz.md5 
939dd9aa1d247882bc3460f19447508e

md5sum differs. Possibly this causes infinite loop.

Partial file is attached in case you want to do further analysis.

So please check.

Thanks and regards,

Original comment by amis...@gmail.com on 16 Dec 2012 at 4:30

Attachments:

• emerging.rules.tar.gz

[GoogleCodeExporter]

Is this still occurring, I am unable to reproduce...

Original comment by Cummin...@gmail.com on 21 Mar 2013 at 2:38

[GoogleCodeExporter]

Ummm. Well it was happening occasionally. But then I had put ulimit so that has 
solved my problem.

As I mentioned in previous messages above it could be some infinite recursion 
somewhere. But I could not verify anything.

Original comment by amis...@gmail.com on 21 Mar 2013 at 2:56

[GoogleCodeExporter]

Couldn't reproduce, will keep an eye out though, for now set a ulimit as a 
workaround.

JJC

Original comment by Cummin...@gmail.com on 21 Mar 2013 at 4:35

• Changed state: WontFix