Bro's http.log needs to be per-interface

[GoogleCodeExporter]

Email conversation with Seth Hall:

"> My next step was going to be replacing httpry with Bro's http.log. But I 
realized last night that there will be a problem in Sguil when right-clicking 
an HTTP transaction and requesting the full pcap. Since we have one http.log 
which could be covering multiple interfaces, Sguil won't know which interface 
to request the pcap from.  I don't have access to a box right now...does 
http.log contain the interface that saw the transaction?  If so, perhaps I can 
tweak Sguil's http-agent to grep for the desired interface. If not, any other 
ideas?

I was about to reply and say that I didn't know what to do, but then I had an 
idea. :)  You can actually use the logging framework to split any of the logs 
like this fairly arbitrarily.  I attached a file named security-onion.bro which 
will do this for you with the http.log and conn.log (turning them into 
http_eth0.log, http_eth1.log, conn_eth0.log, conn_eth1.log, etc.) if the user 
has deployed as a cluster.  If they deploy in standalone mode they will just 
have http.log and conn.log

I recommend dropping the security-onion.bro script into 
<prefix>/share/bro/site/ and adding "@load security-onion" to local.bro."

event bro_init()
        {
        if ( reading_live_traffic() )
                {
                Log::remove_default_filter(HTTP::LOG);
                Log::add_filter(HTTP::LOG, [$name = "http-interfaces",
                                            $path_func(id: Log::ID, path: string, rec: HTTP::Info) = 
                                                { 
                                                local peer = get_event_peer()$descr;
                                                if ( peer in Cluster::nodes && Cluster::nodes[peer]?$interface )
                                                        return cat("http_", Cluster::nodes[peer]$interface);
                                                else
                                                        return "http";
                                                }
                                            ]);

                Log::remove_default_filter(Conn::LOG);
                Log::add_filter(Conn::LOG, [$name = "conn-interfaces",
                                            $path_func(id: Log::ID, path: string, rec: Conn::Info) = 
                                                { 
                                                local peer = get_event_peer()$descr;
                                                if ( peer in Cluster::nodes && Cluster::nodes[peer]?$interface )
                                                        return cat("conn_", Cluster::nodes[peer]$interface);
                                                else
                                                        return "conn";
                                                }
                                            ]);
                }
        }

Original issue reported on code.google.com by doug.bu...@gmail.com on 24 Jan 2012 at 1:49

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Original comment by doug.bu...@gmail.com on 9 May 2012 at 10:43

• Changed title: Bro's http.log needs to be per-interface
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

/usr/bin/fpm -s dir -t deb -n securityonion-bro-security-onion -v 20120511 
/usr/local/share/bro/site/security-onion.bro 
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
Created /home/build/20120511/securityonion-bro-security-onion_20120511_i386.deb

Original comment by doug.bu...@gmail.com on 9 May 2012 at 12:50

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Added the following to security-onion-upgrade.sh:

sed -i 's| |=|g' $CONF
source $CONF
if [ "$VERSION" = "20120508" ]; then
        NEW="20120511"
        echo "**********************************************"   | $LOGGER
        echo "* Upgrading from $VERSION to $NEW."               | $LOGGER
        echo "**********************************************"   | $LOGGER
        DIR="/nsm/backup/$NEW"
        mkdir -p $DIR                                           | $LOGGER
        cd $DIR

    if pgrep httpry>/dev/null; then
        echo "* Stopping old httpry processes."     | $LOGGER
        nsm_sensor_ps-stop --only-httpry
        nsm_sensor_ps-stop --only-httpry-agent
        pkill -f httpry
        echo ""
    fi

        for FILE in securityonion-nsmnow-admin-scripts_20120511_i386.deb securityonion-http-agent_20120511_i386.deb securityonion-bro-security-onion_20120511_i386.deb; do
                echo -n "* Downloading $FILE..."                | $LOGGER
                wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
                if [ $? -eq 1 ]; then
                        echo "FAIL"     | $LOGGER
                        exit 1
                else
                        echo "OK"       | $LOGGER
                fi
        done

        echo -n "* Installing downloaded packages..." | $LOGGER
        dpkg -i *.deb                                           >> $LOG
        if [ $? -eq 1 ]; then
                echo "FAIL"     | $LOGGER
                exit 1
        else
                echo "OK"       | $LOGGER
        fi
    echo ""

    echo "* Updating Bro."         | $LOGGER
    echo "@load security-onion" >> /usr/local/share/bro/site/local.bro
    broctl install      | $LOGGER
    if pgrep -f broctl>/dev/null; then
        broctl restart  | $LOGGER
        sleep 5
        echo ""
    fi

    grep -v "^#" /etc/nsm/sensortab |awk '{print $1}' |while read SENSOR; do
        mv /etc/nsm/$SENSOR/httpry_agent.exclude /etc/nsm/$SENSOR/http_agent.exclude
        mv /etc/nsm/$SENSOR/httpry_agent.conf /etc/nsm/$SENSOR/http_agent.conf
        echo "# LOG_FORMAT" >> /etc/nsm/$SENSOR/http_agent.conf
        echo "# httpry or suricata or bro" >> /etc/nsm/$SENSOR/http_agent.conf
        echo "set LOG_FORMAT bro" >> /etc/nsm/$SENSOR/http_agent.conf
        nsm_sensor_ps-start --sensor-name=$SENSOR --only-http-agent | $LOGGER
    done

        sed -i "s|VERSION=$VERSION|VERSION=$NEW|g" $CONF        | $LOGGER
        echo "* Upgrade to $NEW complete."                      | $LOGGER
        echo 
fi

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:19

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Tested by:
Scott Runnels
Tom De Vries
David Zawdie

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:22

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Published:
http://securityonion.blogspot.com/2012/05/security-onion-20120511-now-available.
html

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:22

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:23

• Changed state: Verified
• Added labels: ****
• Removed labels: ****