Add support for Glastopf

[jatrost]

http://glastopf.org/

[exabrial]

Have this working on ubuntu 14.04, but I don't have a good install script because I'm a shell script noob.

sudo apt-get install python2.7 python-openssl python-gevent libevent-dev python2.7-dev build-essential make python-chardet python-requests python-sqlalchemy python-lxml python-beautifulsoup mongodb python-pip python-dev python-setuptools git php5 php5-dev liblapack-dev gfortran libmysqlclient-dev libxml2-dev libxslt-dev
sudo pip install --upgrade distribute
cd /opt
sudo git clone git://github.com/glastopf/BFR.git
cd BFR
sudo phpize
sudo ./configure --enable-bfr
sudo make && sudo make install

sudo vi /etc/php5/apache2/php.ini
zend_extension = /usr/lib/php5/20121212/bfr.so

sudo service apache2 stop
sudo update-rc.d -f  apache2 remove

cd /opt
git clone https://github.com/rep/hpfeeds.git
cd hpfeeds
sudo python setup.py install

sudo pip install glastopf

cd /opt
sudo mkdir myhoneypot
cd myhoneypot
sudo glastopf-runner

At this point, check your installation by naivating to http://glastopf?user=;DROP TABLE users and you should see glastopf respond with a delightful mysql message. Kill your glastopf server.

Download the registration.sh file from MHN and run it to get a UUID and Secret. Then in your /opt/myhoneypot/glastop.cfg change the hpfeeds section with the secret and uuid as the ident in the hpfeed section:

[hpfeed]
enabled = True
host = mhn
port = 10000
secret = 23490jopij23nj
# channels comma separated
chan_events = glastopf.events
chan_files = glastopf.files
ident = 8fa546cc-2147-11e4-aa64-00163eec95e2

[jgedeon120]

Here is the deploy script that I just created this evening and working. The test system was Ubuntu 12.04.5.

#!/bin/bash

set -e
set -x

if [ $# -ne 2 ]
    then
        echo "Wrong number of arguments supplied."
        echo "Usage: $0 <server_url> <deploy_key>."
        exit 1
fi

server_url=$1
deploy_key=$2
GLASTOPF_HOME=/opt/glastopf

wget $server_url/static/registration.txt -O registration.sh
chmod 0755 registration.sh
# Note: This will export the HPF_* variables
. ./registration.sh $server_url $deploy_key "glastopf"

# Update repository
apt-get update

# Install Prerequisites
apt-get install -y python2.7 python-openssl python-gevent libevent-dev python2.7-dev build-essential make python-chardet python-requests python-sqlalchemy python-lxml python-beautifulsoup mongodb python-pip python-dev python-setuptools git php5 php5-dev liblapack-dev gfortran libmysqlclient-dev libxml2-dev libxslt-dev supervisor
pip install --upgrade distribute

# Install and configure the PHP sandbox
cd /opt
git clone git://github.com/glastopf/BFR.git
cd BFR
phpize
./configure --enable-bfr
make && make install

# Updated php.ini to add bfr.so
BFR_BUILD_OUTPUT=`find /usr/lib/php5/ -type f -name "bfr.so" | awk -F"/" '{print $5}'`
echo "zend_extension = /usr/lib/php5/$BFR_BUILD_OUTPUT/bfr.so" >> /etc/php5/apache2/php.ini

# Stop apache2 and disable it from start up
service apache2 stop
update-rc.d -f  apache2 remove

# Upgrade python-greenlet
pip install --upgrade greenlet

# Install glastopf
pip install glastopf
mkdir -p $GLASTOPF_HOME

# Add the modified glastopf.cfg
cat > $GLASTOPF_HOME/glastopf.cfg <<EOF
[webserver]
host = 0.0.0.0
port = 80
uid = nobody
gid = nogroup
proxy_enabled = False

#Generic logging for general monitoring
[logging]
consolelog_enabled = False
filelog_enabled = True
logfile = log/glastopf.log

[dork-db]
enabled = True
pattern = rfi
#Extracts dorks from a online dorks service operated by The Honeynet Project
mnem_service = True

[hpfeed]
enabled = True
host = $HPF_HOST
port = $HPF_PORT
secret = $HPF_SECRET
# channels comma separated
chan_events = glastopf.events
chan_files = glastopf.files
ident = $HPF_IDENT

[main-database]
#If disabled a sqlite database will be created (db/glastopf.db)
#to be used as dork storage.
enabled = True
#mongodb or sqlalchemy connection string, ex:
#mongodb://localhost:27017/glastopf
#mongodb://james:bond@localhost:27017/glastopf
#mysql://james:bond@somehost.com/glastopf
connection_string = sqlite:///db/glastopf.db

[surfcertids]
enabled = False
host = localhost
port = 5432
user =
password =
database = idsserver

[syslog]
enabled = False
socket = /dev/log

[mail]
enabled = False
# an email notification will be sent only if a specified matched pattern is identified.
# Use the wildcard char *, to be notified every time
patterns = rfi,lfi
user =
pwd =
mail_from =
mail_to =
smtp_host = smtp.gmail.com
smtp_port = 587

[taxii]
enabled = False
host = taxiitest.mitre.org
port = 80
inbox_path = /services/inbox/default/
use_https = False
use_auth_basic = False
auth_basic_username = your_username
auth_basic_password = your_password
use_auth_certificate = False
auth_certificate_keyfile = full_path_to_keyfile
auth_certificate_certfile = full_path_to_certfile
include_contact_info = False
contact_name = ...
contact_email = ...

[misc]
# set webserver banner
banner = Apache/2.0.48
EOF

# Set up supervisor
cat > /etc/supervisor/conf.d/glastopf.conf <<EOF
[program:glastopf]
command=/usr/bin/python /usr/local/bin/glastopf-runner
directory=$GLASTOPF_HOME
stdout_logfile=/var/log/glastopf.out
stderr_logfile=/var/log/glastopf.err
autostart=true
autorestart=true
redirect_stderr=true
stopsignal=QUIT
EOF

supervisorctl update

[exabrial]

@jt6211 This is sort of a general question... is there a way to see the contents of an attack in MHN? It'd be neat to be able to see the SQL the attacker was trying to inject into Glastopf, or the contents of a payload from a Dionaea honetpoint.

[jatrost]

Thanks for posting this here. I am anxious to try it out and get it integrated, but have been swamped with other work projects. @exabrial MHN stores all the data returned from the honey pots in mongodb. Not all this data is exposed in the web app yet, but you should be able to see the data your interested in by querying the "session" and "hpfeed" collections under the "mnemosyne" database. Here is a really basic example:

root@mhn:~# mongo 
MongoDB shell version: 2.6.2
connecting to: test
> use mnemosyne
switched to db mnemosyne
> db.hpfeed.find()
{ "_id" : ObjectId("xxxxxxxxxxxxxxxxxxxx"), "ident" : "XXXXXX-XXXX-XXXXX-XXXXX-XXXXX", "timestamp" : ISODate("2014-05-12T20:40:30.195Z"), "normalized" : false, "payload" : "{\"connection_type\": \"accept\", \"local_host\": \"::ffff:XXX.XXX.XXX.117\", \"connection_protocol\": \"httpd\", \"remote_port\": 51912, \"local_port\": 80, \"remote_hostname\": \"\", \"connection_transport\": \"tcp\", \"remote_host\": \"::ffff:YYY.YYY.YYY.YYY\"}", "channel" : "dionaea.connections", "last_error" : "No normalizer found", "last_error_timestamp" : ISODate("2014-05-12T16:40:30.967Z") }
...

Mongo has python APIs and a powerful javascript based API. After we get Glastopf integrated it would make sense to expose this in the webapp.

[jatrost]

support added here: https://github.com/threatstream/mhn/pull/57.

closing issue.