need help on mysql issue

[sittikhadijah]

Hello i'm newbie in glastopf, would you like to help me ?? i want to store attack information in a MySQL database. but i dont know the configuration steps. my problem is i dont know how to add mysql.py in the glastopf.cfg plug-in section 7.2.2.2 and add the MySQL connection information in the MySQL section 7.2.2.3. i also can't find structures folder in the Glastopf directory. I really need your help. Thanks before.

[johnnykv]

Did you take a look at the configuration file? It has a mysql example.

[glaslos]

@sittikhadijah you are looking at the wrong documentation :)

[sittikhadijah]

am i ? oh, i'm so sorry. so how do i create the glastopf database structure in mysql ? is there the .sql file that i can load in mysql-server or phpmyadmin ? sorry, my english is really bad. :')

[glaslos]

Glastopf takes care of that :)

[sittikhadijah]

oh, so i just have to change the connection_string on glastopf.cfg file as like the example ?

[glaslos]

Exactly.

[sittikhadijah]

pheww, thank you so much for ur help, ah one more thing, actually i separate the honeypot server and the honeypot database server. is that work ?? if yes, there are some additional configuration that i should do ??

[glaslos]

Nope, that should be perfectly fine and a good choice.

[sittikhadijah]

Okay ! thank you so much for your help, maybe later i'll ask some questions if i get stuck in my honeypot. wish me luck for my final project ;)

[sittikhadijah]

i already change glastopf.cfg, and when i started the glastopf, this is the output : Is that okay ? or there is something wrong ?

[glaslos]

You are using sqlite, is that intentional? Has you box connection to the Internet?

[sittikhadijah]

no i don't, i want to use mysql, here is the glastopf.cfg file : am i missing something ?

[johnnykv]

Are you sure your system is able to resolve?

[sittikhadijah]

i think it resolve because i already trying to access mysql-server from client just like this picture :

[johnnykv]

Hmm, looking at your previous screenshot it looks like it works? Only problem seems to be the connection to hpfeeds. Have you checked if Glastopf actually stores data in your mysql db?

[sittikhadijah]

i already check mysql db and the glastopf database is still empty, there is no tables in the inside. maybe i missing some configuration. is there another configuration that i should do ? [hpfeed] part on glastopf.cfg maybe ?

[glaslos]

Could you show us the whole output from Glastopf again? Also point your browser to Glastopf so we can see if it works.

[sittikhadijah]

i'm so sorry, i forgot to change glastopf.cfg in myhoneypot directory, i only change glastopf.cfg in glastopf directory. :D. and now glastopf database isn't empty anymore. here is the whole output from my glastopf : 1.) 2.) 3.) 4.) 5.)

[glaslos]

First, change your MySQL password :)

[glaslos]

For some reason both hpfriends.honeynet.org and mnemosyne.honeycloud.net are not resolving for you.

[glaslos]

Please copy and paste the text from the command line. Screenshots are big and google hates them.

[sittikhadijah]

oh, sorry actually this is for my final project so i just do some research, i use vmware to running my honeypot, so i couldn't copy paste the text to my host. Oke, i'll change MySQL password, so what's next ??

[johnnykv]

Seems like the issue is around here. Accordingly to the screenshots some invalid character is used as query parameter. @sittikhadijah two questions: 1) Which version of sqlalchemy do you have installed? 2) Does this also happen if you use sqlite?

[sittikhadijah]

i use python-sqlalchemy version 0.7.4-1ubuntu0. if i use sqlite the output is just like my first screenshoot on top (https://f.cloud.github.com/assets/4487074/583368/991f4168-c8eb-11e2-819d-ddfd1e3f5b94.png).

[glaslos]

This looks fine to me. You might want to disable hpfeeds and the mnemosyne service. please also test glastopf by pointing your browser to it.

[johnnykv]

These are not critical errors, glastopf is designed to keep operating even if hpfriends/mnemosyne is down. I need you to enable sqlite and trigger the sql error shown in one of your screenshots.

[sittikhadijah]

1. @glaslos i'm sorry, i don't understand "test glastopf by pointing your browser to it." mean, can you explain to me more detail how to test it?
2. @johnnykv how to trigger the sql error ? please give mi some instruction :(

[glaslos]

Open localhost in your browser on you make me and Johnny happy :)

[johnnykv]

Go to the honeypot URL with your browser. We need to be able to reproduce this bug - else we cannot fix it.

On 31/05/2013, at 11.21, sittikhadijah notifications@github.com wrote:

@glaslos i'm sorry, i don't understand "test glastopf by pointing your browser to it." mean, can you explain to me more detail how to test it? @johnnykv how to trigger the sql error ? please give mi some instruction :( — Reply to this email directly or view it on GitHub.

[sittikhadijah]

ooooooohhhh, how stupid i'm -__-" this is the screenshot of my browser when i access localhost : https://f.cloud.github.com/assets/4487074/589777/3e2e71d0-c9cb-11e2-8d3e-443d4321f57f.png and this is my terminal screenshot :

[glaslos]

@johnnykv could the SQL exception originate from an empty table? He is not able to bootstrap from mnemosyne. @sittikhadijah could you run your VM in bridged network mode so you can connect to out bootstrap system?

[sittikhadijah]

actually i have some problem to do that, because when i activate my second network adapter to bridge, and my first network adapter to vmnet, my dns doesn't work because the name server is change to ip 127.0.0.1 so my honeypot database vm could not resolved. so i think both of them can't work at the same time :(

[johnnykv]

@glaslos If glastopf cannot bootstrap from mnemosyne it will fallback to the old method. (just tested this to make sure), which makes me wonder why @sittikhadijah DB is empty after bootstrapping. Also just tested with sqlite on a vm disconnected from the internet and could not reproduce. @sittikhadijah In you last screenshot, the critical error is the one which says something like "You have an error in your SQL syntax; check the manual...." I would really like to know if you can reproduce this error when using sqlite.

[johnnykv]

Ok guys, hang on! I am able to reproduce now.

[johnnykv]

@sittikhadijah please try the latest version on github - i believe this issue is fixed now.

[sittikhadijah]

i already reinstall my glastopf, and these are the output in my terminal when i run it : 2013-06-04 10:53:52,207 (glastopf.modules.handlers.emulators.dork_list.dork_page_generator) Bootstrapping dork database. 2013-06-04 10:53:52,231 (requests.packages.urllib3.connectionpool) Starting new HTTPS connection (1): mnemosyne.honeycloud.net 2013-06-04 10:54:30,903 (glastopf.modules.handlers.emulators.dork_list.mnem_service) Error while communication with mnemosyne: HTTPSConnectionPool(host='mnemosyne.honeycloud.net', port=8282): Max retries exceeded with url: /login (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known) 2013-06-04 10:54:46,504 (glastopf.modules.reporting.auxiliary.log_hpfeeds) Connecting to feed broker. 2013-06-04 10:55:06,513 (glastopf.modules.reporting.auxiliary.log_hpfeeds) Could not connect to hpfeed broker. Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/Glastopf-3.0.8_dev-py2.7.egg/glastopf/modules/reporting/auxiliary/log_hpfeeds.py", line 109, in connect self.socket.connect((self.options["host"], self.options["port"])) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(_args) gaierror: [Errno -5] No address associated with hostname 2013-06-04 10:55:06,521 (glastopf.glastopf) Glastopf started and privileges dropped. 2013-06-04 10:57:12,540 (glastopf.glastopf) 127.0.0.1 requested GET / on localhost 2013-06-04 10:57:12,888 (glastopf.modules.reporting.auxiliary.log_hpfeeds) Connection error: [Errno 9] Bad file descriptor Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/Glastopf-3.0.8_dev-py2.7.egg/glastopf/modules/reporting/auxiliary/log_hpfeeds.py", line 134, in send_data self.socket.send(msgpublish(self.options["ident"], channel, data)) File "/usr/lib/python2.7/socket.py", line 170, in _dummy raise error(EBADF, 'Bad file descriptor') error: [Errno 9] Bad file descriptor 2013-06-04 10:57:12,888 (glastopf.modules.reporting.auxiliary.log_hpfeeds) Connecting to feed broker. 2013-06-04 10:57:12,890 (glastopf.glastopf) 127.0.0.1 requested GET /style.css on localhost 2013-06-04 10:57:13,093 (glastopf.glastopf) 127.0.0.1 requested GET /favicon.ico on localhost 2013-06-04 10:57:13,113 (glastopf.glastopf) 127.0.0.1 requested GET /favicon.ico on localhost 2013-06-04 10:57:32,902 (glastopf.modules.reporting.auxiliary.log_hpfeeds) Could not connect to hpfeed broker. Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/Glastopf-3.0.8_dev-py2.7.egg/glastopf/modules/reporting/auxiliary/log_hpfeeds.py", line 109, in connect self.socket.connect((self.options["host"], self.options["port"])) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(_args) gaierror: [Errno -5] No address associated with hostname Exception in thread Thread-2: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 551, in bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 504, in run self.__target(_self.args, *_self.__kwargs) File "/usr/local/lib/python2.7/dist-packages/Glastopf-3.0.8_dev-py2.7.egg/glastopf/glastopf.py", line 128, in post_processer logger.insert(attack_event) File "/usr/local/lib/python2.7/dist-packages/Glastopf-3.0.8_dev-py2.7.egg/glastopf/modules/reporting/auxiliary/log_hpfeeds.py", line 130, in insert self.send_data(channel, data) File "/usr/local/lib/python2.7/dist-packages/Glastopf-3.0.8_dev-py2.7.egg/glastopf/modules/reporting/auxiliary/log_hpfeeds.py", line 138, in send_data self.socket.send(msgpublish(self.options["ident"], channel, data)) File "/usr/lib/python2.7/socket.py", line 170, in _dummy raise error(EBADF, 'Bad file descriptor') error: [Errno 9] Bad file descriptor

2013-06-04 10:58:51,135 (glastopf.glastopf) 127.0.0.1 requested GET /index1.php?chapter= on localhost 2013-06-04 10:58:51,304 (glastopf.glastopf) 127.0.0.1 requested GET /style.css on localhost

i see there are some error, would you like to explain it to me about the error message ? is there an analysis tools for glastopf ? just like kippo-graph in kippo honeypot.

[johnnykv]

could you post your entire glastopf.cfg? (remove your mysql password, please)

[sittikhadijah]

[webserver]
host = 0.0.0.0
port = 80
uid = nobody
gid = nogroup
proxy_enabled = False

#Generic logging for general monitoring
[logging]
consolelog_enabled = True
filelog_enabled = True
logfile = log/glastopf.log

[dork-db]
enabled = True
pattern = rfi
token_pattern = /\w+
#parameters for clustering of dorks (KMeans)
n_clusters = 10
max_iter = 50
n_init = 20
#Extracts dorks from a online dorks service operated by The Honeynet Project
mnem_service = True

[hpfeed]
enabled = True
host = hpfriends.honeycloud.net
port = 20000
secret = 3wis3l2u5l7r3cew
# channels comma separated
chan_events = glastopf.events
chan_files = glastopf.files
ident = x8yer@hp1

[main-database]
#If disabled a sqlite database will be created (db/glastopf.db)
#to be used as dork storage.
enabled = True
#mongodb or sqlalchemy connection string, ex:
#mongodb://localhost:27017/glastopf
#mongodb://james:bond@localhost:27017/glastopf
#mysql://james:bond@somehost.com/glastopf
connection_string = mysql://glastopf:xxxx@honeypotdb.telkomcloud.com/glastopf

[surfcertids]
enabled = False
host = localhost
port = 5432
user = 
password = 
database = idsserver

[syslog]
enabled = False
socket = /dev/log

[mail]
enabled = False
# an email notification will be sent only if a specified matched pattern is identified.
# Use the wildcard char *, to be notified every time
patterns = rfi,lfi
user =
pwd =
mail_from =
mail_to = 
smtp_host = smtp.gmail.com
smtp_port = 587

[misc]
# set webserver banner
banner = Apache/2.0.48

[glaslos]

Looks like hpfeeds is not failing properly. Anyway, you might want deploy Glastopf in a network with access to the Internet otherwise you are not going to get any malicious traffic.

[sittikhadijah]

actually what is the function of hpfeeds ?

[glaslos]

https://github.com/glastopf/glastopf#hpfeeds

[sittikhadijah]

oh, okay thank you for your help, i've read a blog about glastopf and glasif, is glasif available for the latest version of glastopf ?

[glaslos]

No and not planned for the future.

[sittikhadijah]

oke. thank you very much for your help.

[akanshgulati]

@johnnykv Can anyone share the syntax for connection mysql instead of sqlite. I have a database with the name glastopf, user is root and host is localhost. Where I have to add the password of the root?

[glaslos]

mysql://root:password@localhost/glastopf