search

Security-Onion-Solutions / security-onion

Security Onion 16.04 - Linux distro for threat hunting, enterprise security monitoring, and log management
https://securityonion.net
3.07k stars 522 forks source link

Provide single location for configuring BPF filters #114

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
This is an enhancement request to provide a single location where users can 
input BPF filters to exclude uninteresting traffic.  For example, exclude 
IPSEC-encrypted traffic with the filter "not ip proto 50".  This will help 
maximize disk space and perhaps improve performance by minimizing extraneous 
data capture.

Here is the solution from Karlios from the mailing list:

i use this configuration

nano /etc/nsm/"sensor_name"/bpf.conf
                add your BPF filters
       eg.
               (
               (host x.x.x.x) or
               (host x.x.x.x)
               )

       chown sguil:sguil /etc/nsm/"sensor_name"/bpf.conf
       chmod 644 /etc/nsm/"sensor_me"/bpf.conf

edit file:
       nano /etc/nsm/"sensor_name"/sensor.conf
       from SNORT_OPTIONS="-m 112"  to SNORT_OPTIONS="-m 112 -F /etc/
nsm/"sensor_name"/bpf.conf"
       or
sed -i "s|^SNORT_OPTIONS=\"-m 112\".*$|SNORT_OPTIONS=\"-m 112 -F /etc/
nsm/"sensor_name"/bpf.conf\"|g" /etc/nsm/"sensor_name"/sensor.conf

BPF filter writing examples http://seclists.org/snort/2010/q3/468

p.s. Maybe  it would be nice to have bpf.conf (empty one) in default
securityonion distribution.

Original issue reported on code.google.com by mpilk...@gmail.com on 13 Jul 2011 at 2:25

GoogleCodeExporter commented 9 years ago 
For creating new sensors, /usr/local/sbin/nsm_sensor_add needs to be modified 
as follows (untested):
sed -i 's|SNORT_OPTIONS="${SNORT_OPTIONS}"|SNORT_OPTIONS="${SNORT_OPTIONS} -F 
${SENSOR_CONF_DIR}/${SENSOR_NAME}/bpf.conf"|g' /usr/local/sbin/nsm_sensor_add

It also needs to create the empty bpf.conf.

Original comment by doug.bu...@gmail.com on 13 Jul 2011 at 10:49

GoogleCodeExporter commented 9 years ago 
To create empty bpf.conf file edit /usr/local/sbin/nsm_sensor_add

add the code below one of the sections ending with EOF_smf_smf  e.g. 
EOF_PCAP_CONF

THE_TIME=$(date)
cat >/etc/nsm/$SENSOR_NAME/bpf.conf << EOF_BPF_CONF
EOF_BPF_CONF

p.s. 
I think /usr/local/sbin/nsm_sensor_edit should also be edited same way as 
/usr/local/sbin/nsm_sensor_add script

Need testing

Original comment by karolis....@gmail.com on 13 Jul 2011 at 11:53

GoogleCodeExporter commented 9 years ago 
Snort and Suricata use -F but daemonlogger uses -f.

Original comment by doug.bu...@gmail.com on 8 Dec 2011 at 6:03

GoogleCodeExporter commented 9 years ago 
Issue 221 has been merged into this issue.

Original comment by doug.bu...@gmail.com on 8 Feb 2012 at 7:45

GoogleCodeExporter commented 9 years ago 
Emailed seth hall on this to see if we could get something a little more sane 
for 

bro.  Currently bro supports BPF, but in its' own format:

Add this to local.bro:

redef PacketFilter::all_packets = F;
redef capture_filters += {
    ["packets-like-this"] = "ip or not ip", }; redef restrict_filters += {
    ["no-data-like-this"] = "not net 1.2.3.0/24",
    ["also-no-data-like-this"] = "not port 31337", };

And you will end up with this (essentially):
    "(ip or not ip) and (not net 1.2.3.0/24 and not port 31337)"

When I hear back on bro I will update / link in their ticket.

Original comment by liam.ran...@gmail.com on 8 Feb 2012 at 10:55

GoogleCodeExporter commented 9 years ago 
Bro supports a similar syntax:

$ bro -h
<snip>
    -f|--filter <filter>           | tcpdump filter

Original comment by liam.ran...@gmail.com on 9 Feb 2012 at 2:15

GoogleCodeExporter commented 9 years ago 
After speaking to Seth about this further they would prefer that we use a bro 
script to implement the bro bpf filter, syntax:

redef cmd_line_bpf_filter = "whatever your filter is";

Original comment by liam.ran...@gmail.com on 9 Feb 2012 at 2:42

GoogleCodeExporter commented 9 years ago 
I think the script makes the most sense for now.  2.1 is going to have an 
overhauled packet-filter framework which will have easier mechanisms for 
filtering but an approach very similar to this should still be available.

Original comment by seth.h...@gmail.com on 10 Feb 2012 at 4:30

GoogleCodeExporter commented 9 years ago 
I'm working on this issue now and have completed the initial changes for Snort, 
Suricata, and Daemonlogger.

Since Bro has one config for all interfaces in the box, I don't think we can 
accurately have one BPF file that would work for Bro and all of the 
per-interface instances of Snort/Suricata/Daemonlogger at the same time.  So 
we'll just create a BPF page on the Wiki with the above documentation on Bro 
BPF configuration.

Original comment by doug.bu...@gmail.com on 27 Mar 2012 at 9:01

GoogleCodeExporter commented 9 years ago 
I discovered this morning that Suricata's afpacket mode
currently does not support BPF.  I've created a Feature Request for
that to be added.

The good news is that I've tested our new BPF update successfully with
Snort and daemonlogger.

Original comment by doug.bu...@gmail.com on 28 Mar 2012 at 2:55

GoogleCodeExporter commented 9 years ago 
Updated code in /usr/local/sbin/nsm_sensor_ps-start:

    # Start IDS Engine with unified2 output
    # Determine whether to use Suricata or Snort (default)
    if grep -i "suricata" /etc/nsm/securityonion.conf >/dev/null
    then
        # Start Suricata
        [ -z "$SKIP_SNORT_ALERT" ] && process_start "suricata" "--user $SENSOR_USER --group $SENSOR_GROUP -c /etc/nsm/$SENSOR/suricata.yaml --af-packet=$SENSOR_INTERFACE_SHORT --runm
ode=autofp -F /etc/nsm/$SENSOR/bpf.conf -l $SENSOR_LOG_DIR " 
"$PROCESS_PID_DIR/$SENSOR/suricata.pid" "$PROCESS_LOG_DIR/$SENSOR/suricata.log" 
"suricata (alert data)"

    else    
        # Start Snort (default)
        [ -z "$SKIP_SNORT_ALERT" ] && process_start "snort" "-u $SENSOR_USER -g $SENSOR_GROUP -c $SNORT_CONFIG --daq afpacket -i $SENSOR_INTERFACE_SHORT -F /etc/nsm/$SENSOR/bpf.conf 
-l $SENSOR_LOG_DIR $SNORT_OPTIONS" "$PROCESS_PID_DIR/$SENSOR/snortu.pid" 
"$PROCESS_LOG_DIR/$SENSOR/snortu.log" "snort (alert data)"
    fi
    # start barnyard2
    [ -z "$SKIP_BARNYARD2" ] && process_start "barnyard2" "-c $BARNYARD2_CONFIG -d $SENSOR_LOG_DIR -f snort.unified2 -w $BARNYARD2_WALDO $BARNYARD2_OPTIONS" "$PROCESS_PID_DIR/$SENSOR/bar
nyard2.pid" "$PROCESS_LOG_DIR/$SENSOR/barnyard2.log" "barnyard2 (spooler, 
unified2 format)"

    # start sancp
    [ -z "$SKIP_SANCP" ] && process_start "sancp" "-d $SENSOR_LOG_DIR/sancp -i $SENSOR_INTERFACE_SHORT -c $SANCP_CONFIG -u $SENSOR_USER -g $SENSOR_GROUP $SANCP_OPTIONS" "$PROCESS_PID_DIR
/$SENSOR/sancp.pid" "$PROCESS_LOG_DIR/$SENSOR/sancp.log" "sancp (session data)"

        # start pads
        [ -z "$SKIP_PADS" ] && process_start "pads" "-i $SENSOR_INTERFACE_SHORT -c $PADS_CONFIG -u $SENSOR_USER -g $SENSOR_GROUP $PADS_OPTIONS" "$PROCESS_PID_DIR/$SENSOR/pads.pid" "$PROCESS_
LOG_DIR/$SENSOR/pads.log" "pads (asset info)"
        [ -z "$SKIP_PADS_AGENT" ] && process_start "pads_agent.tcl" "-c $PADS_AGENT_CONFIG" "$PROCESS_PID_DIR/$SENSOR/pads_agent.pid" "$PROCESS_LOG_DIR/$SENSOR/pads_agent.log" "pads_agent (s
guil)"

    # start daemonlogger
    TODAY=$(date $DATE_OPTIONS "+%Y-%m-%d")      #-u option sets TZ to GMT
    if [ ! -d "$SENSOR_LOG_DIR/dailylogs/$TODAY" ]
    then
        mkdir -p $SENSOR_LOG_DIR/dailylogs/$TODAY
        chown $SENSOR_USER:$SENSOR_GROUP $SENSOR_LOG_DIR/dailylogs/$TODAY
        chmod 775 $SENSOR_LOG_DIR/dailylogs/$TODAY
    fi
    [ -z "$SKIP_DAEMONLOGGER" ] && process_start "daemonlogger" "-u $SENSOR_USER -g $SENSOR_GROUP -i $SENSOR_INTERFACE_SHORT -f /etc/nsm/$SENSOR/bpf.conf -l $SENSOR_LOG_DIR/dailylogs/$TO
DAY -n snort.log -s 134217728" "$PROCESS_PID_DIR/$SENSOR/daemonlogger.pid" 
"$PROCESS_LOG_DIR/$SENSOR/daemonlogger.log" "daemonlogger (full packet data)"

Original comment by doug.bu...@gmail.com on 28 Mar 2012 at 5:06

GoogleCodeExporter commented 9 years ago 

Updated code in /usr/local/sbin/nsm_sensor_ps-restart:

    # restart the IDS engine
        if grep -i "suricata" /etc/nsm/securityonion.conf >/dev/null
    then
        [ -z "$SKIP_SNORT_ALERT" ] && $ACTION "suricata" "--user $SENSOR_USER --group $SENSOR_GROUP -c /etc/nsm/$SENSOR/suricata.yaml --af-packet=$SENSOR_INTERFACE_SHORT --runmode=au
tofp -F /etc/nsm/$SENSOR/bpf.conf -l $SENSOR_LOG_DIR" 
"$PROCESS_PID_DIR/$SENSOR/suricata.pid" "$PROCESS_LOG_DIR/$SENSOR/suricata.log" 
"suricata (alert data)"
    else
        [ -z "$SKIP_SNORT_ALERT" ] && $ACTION "snort" "-u $SENSOR_USER -g $SENSOR_GROUP -c $SNORT_CONFIG --daq afpacket -i $SENSOR_INTERFACE_SHORT -F /etc/nsm/$SENSOR/bpf.conf -l $SE
NSOR_LOG_DIR $SNORT_OPTIONS" "$PROCESS_PID_DIR/$SENSOR/snortu.pid" 
"$PROCESS_LOG_DIR/$SENSOR/snortu.log" "snort (alert data)"
    fi

    # restart barnyard2
    [ -z "$SKIP_BARNYARD2" ] && $ACTION "barnyard2" "-c $BARNYARD2_CONFIG -d $SENSOR_LOG_DIR -f snort.unified2 -w $BARNYARD2_WALDO $BARNYARD2_OPTIONS" "$PROCESS_PID_DIR/$SENSOR/barnyard2
.pid" "$PROCESS_LOG_DIR/$SENSOR/barnyard2.log" "barnyard2 (spooler, unified2 
format)"

    # restart sancp
    [ -z "$SKIP_SANCP" ] && $ACTION "sancp" "-d $SENSOR_LOG_DIR/sancp -i $SENSOR_INTERFACE_SHORT -c $SANCP_CONFIG -u $SENSOR_USER -g $SENSOR_GROUP $SANCP_OPTIONS" "$PROCESS_PID_DIR/$SENS
OR/sancp.pid" "$PROCESS_LOG_DIR/$SENSOR/sancp.log" "sancp (session data)"

    # restart pads
    [ -z "$SKIP_PADS" ] && $ACTION "pads" "-i $SENSOR_INTERFACE_SHORT -c $PADS_CONFIG -u $SENSOR_USER -g $SENSOR_GROUP $PADS_OPTIONS" "$PROCESS_PID_DIR/$SENSOR/pads.pid" "$PROCESS_LOG_DI
R/$SENSOR/pads.log" "pads (asset info)"
    [ -z "$SKIP_PADS_AGENT" ] && $ACTION "pads_agent.tcl" "-c $PADS_AGENT_CONFIG" "$PROCESS_PID_DIR/$SENSOR/pads_agent.pid" "$PROCESS_LOG_DIR/$SENSOR/pads_agent.log" "pads_agent (sguil)"

    # restart daemonlogger
    if [ -z "$SKIP_DAEMONLOGGER" ]
    then
        TODAY=$(date $DATE_OPTIONS "+%Y-%m-%d")      #-u option sets TZ to GMT
        if [ ! -d "$SENSOR_LOG_DIR/dailylogs/$TODAY" ]
        then
            mkdir -p $SENSOR_LOG_DIR/dailylogs/$TODAY
            chown $SENSOR_USER:$SENSOR_GROUP $SENSOR_LOG_DIR/dailylogs/$TODAY
            chmod 775 $SENSOR_LOG_DIR/dailylogs/$TODAY
        fi

        $ACTION_PCAP "daemonlogger" "-u $SENSOR_USER -g $SENSOR_GROUP -i $SENSOR_INTERFACE_SHORT -f /etc/nsm/$SENSOR/bpf.conf -l $SENSOR_LOG_DIR/dailylogs/$TODAY -n snort.log -s 1342
17728" "$PROCESS_PID_DIR/$SENSOR/daemonlogger.pid" 
"$PROCESS_LOG_DIR/$SENSOR/daemonlogger.log" "daemonlogger (full packet data)"
    fi

Original comment by doug.bu...@gmail.com on 28 Mar 2012 at 5:06

GoogleCodeExporter commented 9 years ago 
Added the following to security-onion-upgrade.sh:

sed -i 's| |=|g' $CONF
source $CONF
if [ "$VERSION" = "20120326" ]; then
        NEW="20120329"
        echo "**********************************************"   | $LOGGER
        echo "* Upgrading from $VERSION to $NEW."               | $LOGGER
        echo "**********************************************"   | $LOGGER
        DIR="/nsm/backup/$NEW"
        mkdir -p $DIR                                           | $LOGGER
        cd $DIR

        for FILE in securityonion-nsmnow-admin-scripts_20120329_i386.deb; do
                echo -n "* Downloading $FILE..."                | $LOGGER
                wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
                if [ $? -eq 1 ]; then
                        echo "FAIL"     | $LOGGER
                        exit 1
                else
                        echo "OK"       | $LOGGER
                fi
        done

        echo -n "* Installing downloaded packages..." | $LOGGER
        dpkg -i *.deb                                           >> $LOG
        if [ $? -eq 1 ]; then
                echo "FAIL"     | $LOGGER
                exit 1
        else
                echo "OK"       | $LOGGER
        fi

        SENSORS=`grep -v "^#" /etc/nsm/sensortab |awk '{print $1}'`
        for SENSORNAME in $SENSORS; do
                echo "* Creating /etc/nsm/$SENSORNAME/bpf.conf if it doesn't already exist"       | $LOGGER
        touch /etc/nsm/"$SENSORNAME"/bpf.conf
        done

        sed -i "s|VERSION=$VERSION|VERSION=$NEW|g" $CONF        | $LOGGER
        echo "* Upgrade to $NEW complete."                      | $LOGGER
        echo 
fi

Original comment by doug.bu...@gmail.com on 28 Mar 2012 at 5:11

GoogleCodeExporter commented 9 years ago

Original comment by doug.bu...@gmail.com on 28 Mar 2012 at 5:11

GoogleCodeExporter commented 9 years ago 
Turned over to testing:

Security Onion Testers,

Security Onion 20120328 is ready for testing!  This update should resolve the 
following issues:
http://code.google.com/p/security-onion/issues/detail?id=114
http://code.google.com/p/security-onion/issues/detail?id=224
http://code.google.com/p/security-onion/issues/detail?id=242
http://code.google.com/p/security-onion/issues/detail?id=243

Please only test on VMs that can be snapshotted.

Please test/verify the following:

- Start with a VM with the latest Security Onion and run Setup (choosing Snort 
- Suricata afpacket mode currently doesn't support bpf) so that we can simulate 
an in-place upgrade

- Run the in-place upgrade (should install new package and create 
/etc/nsm/HOSTNAME-INTERFACE/bpf.conf):
sudo -i "curl -L 
http://sourceforge.net/projects/security-onion/files/20120329/security-onion-upg
rade.sh > ~/security-onion-upgrade.sh && bash ~/security-onion-upgrade.sh"

- Add a BPF to /etc/nsm/HOSTNAME-INTERFACE/bpf.conf like the following (for 
testmyids.com):
not host 217.160.51.31

- Run "sudo nsm_sensor_ps-restart" to restart Snort and daemonlogger

- Verify that snort doesn't alert on "curl http://testmyids.com" anymore and 
that daemonlogger didn't record any packets for that destination

- run Setup to simulate a new install

- Run the same test as above.

- Verify issues 224, 242, and 243 are fixed as well

- Anything else I didn't think of

Thanks in advance for your time and effort!

Original comment by doug.bu...@gmail.com on 28 Mar 2012 at 5:37

GoogleCodeExporter commented 9 years ago 
Tested by:
Craig Shannon
Scott Runnels

Original comment by doug.bu...@gmail.com on 29 Mar 2012 at 9:19

GoogleCodeExporter commented 9 years ago 
Published:
http://securityonion.blogspot.com/2012/03/security-onion-20120329-now-available.
html

Original comment by doug.bu...@gmail.com on 29 Mar 2012 at 10:03