search

ZerBea / hcxtools

A small set of tools to convert packets from capture files to hash files for use with Hashcat or John the Ripper.
MIT License
1.99k stars 391 forks source link

Feature Request: Verbose Listing of hash file content #121

Closed zhovner closed 4 years ago

zhovner commented 4 years ago

Hi, ZerBea and thank you for your great work.

I want to look closer at all PMKID in pcap dump. For example I want to see MAC addresses, raw PMKID value and so on.

For example it can look like this:

SSID: MyWiFi
BSSID: 64:d1:54:a1:6c:ab (Routerboard)
STA: 98:01:a7:8e:6d:f8 (Apple) 
RSN PMKID: 68d0646e5a996de5893dc352f6c5b3c2

Hashcat 16800 new format:
f6b117167aeaae3949e594c0ee905789:78b21317e1ce:d0f88c32a9c3

Hashcat 16800 old format:
68d0646e5a996de5893dc352f6c5b3c2*64d154a16fa0*2cd05a754306*6d6172743134

Hashcat 16801 format:
79f7ba04f290b28e64eb451124f4c539:64d154a16fa0:d4f46f8346c0

Currently I use Wireshark to manually check every packet. Maybe you know some existing tool to do this? Or we need to patch hcxpcaptool to add verbose output for PMKID's?

ZerBea commented 4 years ago

hash mode 2500/2501 (hccapx) and 16800/16801 (PMKID) are deprecated. This old formats are combined to new hash mode 22000/22001 starting with latest git head of hashcat, JtR and hcxtools. Hash format 22000 is no longer binary and this hash lines can be posted in forums, or where you want. You can run simple bash commands (awk, sed, cut, cat, ...) on them to show all what you like. Additional I started to code hcxhashtool. It will provide additional features, which can't be done by bash commands. After a medium period I'm going to remove all deprecated 250x/1680x based tools and I think, hashcat and JtR will follow, because we developed this new format together. You can read more about the new has line here: https://github.com/hashcat/hashcat/issues/1816

ZerBea commented 4 years ago

BTW: This format is a nice idea: SSID: MyWiFi BSSID: 64:d1:54:a1:6c:ab (Routerboard) STA: 98:01:a7:8e:6d:f8 (Apple) RSN PMKID: 68d0646e5a996de5893dc352f6c5b3c2

I think about it, to add this to new hcxhashtool.

zhovner commented 4 years ago

hash mode 2500/2501 (hccapx) and 16800/16801 (PMKID) are deprecated

Yeah, but it still popular on some online services like gpuhash.me. So I would like to have an option that will print all possible formats of PMKID hashes sorted by SSID in stdout to simply copy&paste right one without opening any files. I don't suggest to change any logic in file converting, but only to add option that print verbose and clearly readable output to stdout. For example -VV to double verbosity.

Hash format 22000 is no longer binary and this hash lines can be posted in forums, or where you want. You can run simple bash commands (awk, sed, cut, cat, ...)

Right, but it required too many actions. And now I think how can I help: write a new python script that parse 22000 in human readable format with MAC vendor database or just wait when you add this feature to hcxhashtool because my C skills is poor :)

zhovner commented 4 years ago

Also whoismac does not support 22000 format: 

$ whoismac -p WPA*01*71d3d11f7cf836e236889c5fcff17c8b*64d154a16fa0*fc2a9c90e7e9*6d6172743134***
error hashline wrong format

But support old one: 

$ whoismac -p 79f7ba04f290b28e64eb451124f4c539:64d154a16fa0:d4f46f8346c0:4d7957694669

ESSID..: MyWiFi
MAC_AP.: 64d154a16fa0
VENDOR.: Routerboard.com
MAC_STA: d4f46f8346c0
VENDOR.: Apple, Inc.
ZerBea commented 4 years ago

Latest git head support it now, too: $ whoismac -p "WPA*01*71d3d11f7cf836e236889c5fcff17c8b*64d154a16fa0*fc2a9c90e7e9*6d6172743134***"

ESSID..: mart14 MAC_AP.: 64d154a16fa0 VENDOR.: Routerboard.com MAC_STA: fc2a9c90e7e9 VENDOR.: Apple, Inc.

BTW: hashcat 6.0.0 will run 22000 as default and mark 250x/1680x as deprecated, too, because this format has many advantages.

I don't think anyone will stick with the old format any longer...

Also, some new converter tools support this new format, too: https://github.com/s77rt/multicapconverter

zhovner commented 4 years ago

Latest git head support it now, too

Thank you!

I cannot wait and made my silly python wrapper for fancy output look just like I dream: pmkidlist.py It's kinda dirty, because creates temporary files and run binaries in a background, but it do exactly what I want.

Usage is pretty simple pmkidlist.py dump.pcap:

image

I also change whoismac little bit to make their output more compact, vendor name and MAC address now in one line. I don't even try to PR this because of dirty code.

ZerBea commented 4 years ago

Great, your dump is looking fine. Maybe I can still add a similar function (for new hashmode 2200x, only) before release of hcxtools v6.0.0, too. There is still a lot of work to do.

Anyway, we're dropping support for 250x (hccapx) and 1680x step by step.

BTW: Until I'm going to remove hcxmactool, it can be used to convert existing 2500 (hccapx) and 16800 files to 22000. Currently we are testing the new hash mode on big PMKID&EAPOL hash files and the advantages are enormous (reuse PBKDF2 over PMKID and EAPOL). $ hashcat --version v5.1.0-1547-g8e0f976c

JtR will follow, soon: https://github.com/magnumripper/JohnTheRipper/issues/4183 Goal is that the main hash recovery tools (JtR and hascat) running the same potfile and hash file entries on PMKID&EAPOL.

ZerBea commented 4 years ago

added this feature to hcxhashtool: --info=file : output detailed information about content of hash file --info=stdout : stdout output detailed information about content of hash file

example 1; print detailed info to stdout $ hcxhastool -i test.22000 --info=stdout

example 2; print detailed info to file $ hcxhastool -i test.22000 --info=test.info.txt

This format can be used to post hashes as git comments or in forum threads.

If you need additional information fields (WPA type, authorized, not authorized, ANONCE, SNONCE), please let me know. If not, please test and close this feature request.

zhovner commented 4 years ago

I'm not sure that I do it right, but It not work for me:

$ ./hcxpcapngtool -o /tmp/k22000.txt ~/bettercap-wifi-handshakes.pcap
reading from bettercap-wifi-handshakes.pcap...

summary capture file
--------------------
file name..............................: bettercap-wifi-handshakes.pcap
version (pcap/cap).....................: 2.4 (very basic format without any additional information)
timestamp minimum (GMT)................: 16.12.2019 03:35:08
timestamp maximum (GMT)................: 04.01.2020 05:07:03
link layer header type.................: DLT_IEEE802_11_RADIO (127)
endianess (capture system).............: little endian
packets inside.........................: 550
frames with correct FCS................: 550
BEACON.................................: 214
PROBERESONSE...........................: 43
ASSOCIATIONREQUEST.....................: 12
EAPOL messages (total).................: 281
EAPOL M1 messages......................: 265
EAPOL M2 messages......................: 9
EAPOL M3 messages......................: 7
PMKID..................................: 237
PMKID (useless)........................: 9
PMKID written to combi hash file.......: 9

$ cat /tmp/k22000.txt
WPA*01*78837cb042dcb37018d54a3ecfae57d0*3c9872726c1a*9801a78e67f9*4d4754535f47504f4e5f36303037***
WPA*01*dbf49d71e1408a9439c38d58572ad837*64d154a16fa0*0026bb08e356*6d6172743134***
WPA*01*52e394b1d065fc8faed98417b4fea35a*64d154a16fa0*240a64863b0b*6d6172743134***
WPA*01*68d0646e5a996de5893dc352f6c5b3c2*64d154a16fa0*2cd05a754306*6d6172743134***
WPA*01*237c01de36e132c2e260869906f93686*64d154a16fa0*b853ace1f5eb*6d6172743134***
WPA*01*342c9e460838b8530cdb55cf2bb1e76d*64d154a16fa0*b88d120f4654*6d6172743134***
WPA*01*79f7ba04f290b28e64eb451124f4c539*64d154a16fa0*d4f46f8346c0*6d6172743134***
WPA*01*71d3d11f7cf836e236889c5fcff17c8b*64d154a16fa0*fc2a9c90e7e9*6d6172743134***
WPA*01*f6b117167aeaae3949e594c0ee905789*78b21317e1ce*d0f88c32a9c3*4d4754535f47504f4e5f37363432***

$ ./hcxhashtool -i /tmp/k22000.txt --info=stdout

OUI information file...: /Users/a/.hcxtools/oui.txt
OUI entires............: 27381
ZerBea commented 4 years ago

latest git head working fine, here:

$ hcxhashtool -i k22000.txt --info=stdout
SSID......: MGTS_GPON_6007
MAC_AP....: 3c9872726c1a (Sercomm Corporation.)
MAC_CLIENT: 9801a78e67f9 (Apple, Inc.)
PMKID.....: 78837cb042dcb37018d54a3ecfae57d0
HASHLINE..: WPA*01*78837cb042dcb37018d54a3ecfae57d0*3c9872726c1a*9801a78e67f9*4d4754535f47504f4e5f36303037***

SSID......: mart14
MAC_AP....: 64d154a16fa0 (Routerboard.com)
MAC_CLIENT: 0026bb08e356 (Apple, Inc.)
PMKID.....: dbf49d71e1408a9439c38d58572ad837
HASHLINE..: WPA*01*dbf49d71e1408a9439c38d58572ad837*64d154a16fa0*0026bb08e356*6d6172743134***

SSID......: mart14
MAC_AP....: 64d154a16fa0 (Routerboard.com)
MAC_CLIENT: 240a64863b0b (AzureWave Technology Inc.)
PMKID.....: 52e394b1d065fc8faed98417b4fea35a
HASHLINE..: WPA*01*52e394b1d065fc8faed98417b4fea35a*64d154a16fa0*240a64863b0b*6d6172743134***

SSID......: mart14
MAC_AP....: 64d154a16fa0 (Routerboard.com)
MAC_CLIENT: 2cd05a754306 (Liteon Technology Corporation)
PMKID.....: 68d0646e5a996de5893dc352f6c5b3c2
HASHLINE..: WPA*01*68d0646e5a996de5893dc352f6c5b3c2*64d154a16fa0*2cd05a754306*6d6172743134***

SSID......: mart14
MAC_AP....: 64d154a16fa0 (Routerboard.com)
MAC_CLIENT: b853ace1f5eb (Apple, Inc.)
PMKID.....: 237c01de36e132c2e260869906f93686
HASHLINE..: WPA*01*237c01de36e132c2e260869906f93686*64d154a16fa0*b853ace1f5eb*6d6172743134***

SSID......: mart14
MAC_AP....: 64d154a16fa0 (Routerboard.com)
MAC_CLIENT: b88d120f4654 (Apple, Inc.)
PMKID.....: 342c9e460838b8530cdb55cf2bb1e76d
HASHLINE..: WPA*01*342c9e460838b8530cdb55cf2bb1e76d*64d154a16fa0*b88d120f4654*6d6172743134***

SSID......: mart14
MAC_AP....: 64d154a16fa0 (Routerboard.com)
MAC_CLIENT: d4f46f8346c0 (Apple, Inc.)
PMKID.....: 79f7ba04f290b28e64eb451124f4c539
HASHLINE..: WPA*01*79f7ba04f290b28e64eb451124f4c539*64d154a16fa0*d4f46f8346c0*6d6172743134***

SSID......: mart14
MAC_AP....: 64d154a16fa0 (Routerboard.com)
MAC_CLIENT: fc2a9c90e7e9 (Apple, Inc.)
PMKID.....: 71d3d11f7cf836e236889c5fcff17c8b
HASHLINE..: WPA*01*71d3d11f7cf836e236889c5fcff17c8b*64d154a16fa0*fc2a9c90e7e9*6d6172743134***

SSID......: MGTS_GPON_7642
MAC_AP....: 78b21317e1ce (DWnet Technologies(Suzhou) Corporation)
MAC_CLIENT: d0f88c32a9c3 (Motorola (Wuhan) Mobility Technologies Communication Co., Ltd.)
PMKID.....: f6b117167aeaae3949e594c0ee905789
HASHLINE..: WPA*01*f6b117167aeaae3949e594c0ee905789*78b21317e1ce*d0f88c32a9c3*4d4754535f47504f4e5f37363432***

OUI information file...: /home/zerobeat/.hcxtools/oui.txt
OUI entires............: 27383
total lines read.......: 9
valid hash lines.......: 9
PMKID hash lines.......: 9
PMKID written..........: 9
ZerBea commented 4 years ago

If you compare your status: OUI information file...: /Users/a/.hcxtools/oui.txt OUI entires............: 27381

with mine: OUI information file...: /home/zerobeat/.hcxtools/oui.txt OUI entires............: 27383 total lines read.......: 9 valid hash lines.......: 9 PMKID hash lines.......: 9 PMKID written..........: 9

you'll see, that you didn't read something.

ZerBea commented 4 years ago

have you tested it in on a Linux system or on macOS?

zhovner commented 4 years ago

hcxpcapngtool is working correctly now, but I still have no luck with hcxhashtool

$ ./hcxhashtool -i ./test.22000 --info=stdout

OUI information file...: /Users/a/.hcxtools/oui.txt
OUI entires............: 27381
filter by ESSID len min: 0
filter by ESSID len max: 32

$ cat ./test.22000

WPA*02*85ef1323a78904e01f29edf927ed50d8*10feed9bd562*78abbb898296*6d6f6c6f6b6f*9df13312fdf2e37952e9dbea0db72ddd78a4d11e28eba902c575044312362810*0103007502010a000000000000000000015537698ebfe1bf713bcc5240bb8acd3e5baf6ecc4e4f6c0b6796f7fd62a04d68000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020000*00
WPA*01*78837cb042dcb37018d54a3ecfae57d0*3c9872726c1a*9801a78e67f9*4d4754535f47504f4e5f36303037***
WPA*01*dbf49d71e1408a9439c38d58572ad837*64d154a16fa0*0026bb08e356*6d6172743134***
WPA*01*52e394b1d065fc8faed98417b4fea35a*64d154a16fa0*240a64863b0b*6d6172743134***
ZerBea commented 4 years ago

It looks like macOS doesn't like the Linux stdout way. Please try: $ ./hcxhashtool -i ./test.22000 --info=testfile.txt $ cat --info=testfile.txt

zhovner commented 4 years ago

No result even with --info=file.txt, file not created.

zhovner commented 4 years ago

I've been playing with hcxpcapngtool and it works great on macOS. Now I'm thinking why not combine hcxhashtool and whoismac into one hcxpcapngtool tool? My first idea is to get plaintext hash line next to clearly readable parsed SSID and MAC in one command.

It can look like this: 

hcxpcapngtool -p (which means print) mydump.pcapng

SSID......: mart14
MAC_AP....: 64d154a16fa0 (Routerboard.com)
MAC_CLIENT: fc2a9c90e7e9 (Apple, Inc.)
PMKID.....: 71d3d11f7cf836e236889c5fcff17c8b
HASHLINE..: WPA*01*71d3d11f7cf836e236889c5fcff17c8b*64d154a16fa0*fc2a9c90e7e9*6d6172743134***

SSID......: MGTS_GPON_7642
MAC_AP....: 78b21317e1ce (DWnet Technologies(Suzhou) Corporation)
MAC_CLIENT: d0f88c32a9c3 (Motorola (Wuhan) Mobility Technologies Communication Co., Ltd.)
PMKID.....: f6b117167aeaae3949e594c0ee905789
HASHLINE..: WPA*01*f6b117167aeaae3949e594c0ee905789*78b21317e1ce*d0f88c32a9c3*4d4754535f47504f4e5f37363432***

So no need in temporary files and many commands to run. I think this will be the most popular option for those who send hashes somewhere else, for script kiddies like me :)

I spoke with guys from onlinehashcrack.com and they promise me to add 22000 plaintext format support in text field input. I will also try to force all public services to add 220000 support :)

ZerBea commented 4 years ago

No. The Linux philosophy (an mine, too) is: Write programs that do one thing and do it well. Write programs to work together. https://en.wikipedia.org/wiki/Unix_philosophy

hcxdumptool -> WiFi part (fast without additional stuff, able to run headless) hcxpcapngtool -> conversion (fast without additional stuff, able to run headless) hcxhashtool -> provide filter / info about content of hashfile / pre-processor for hashcat/JtR hcxpsktool -> provide information based on MAC and ESSID wlancap2wpasec -> upload to data base hcxwltool -> provide word list functions which other tools don't provide

All other hcxtools are deprecated and I'm going to remove them, soon (when hashcat and JtR remove that old formats).

BTW: If you follow git history, you'll see that I started to move whoismac functions to hcxhashtool.

ZerBea commented 4 years ago

"I think this will be the most popular option for those who send hashes somewhere else,..." No. A pcapng file contain much more (useful) information than a single hashline.

ZerBea commented 4 years ago

added to hcxhashtool. Example: get cap file from here: https://hashcat.net/forum/thread-6661-post-47091.html#pid47091

$ hcxpcapngtool -o test.22000 hashcat.cap --ignore-ie
reading from hashcat.cap...
summary capture file
file name..............................: hashcat.cap
version (pcap/cap).....................: 2.4 (very basic format without any additional information)
timestamp minimum (GMT)................: 01.01.1970 01:00:00
timestamp maximum (GMT)................: 01.01.1970 01:00:00
link layer header type.................: DLT_IEEE802_11 (105)
endianess (capture system).............: little endian
packets inside.........................: 3
packets with zeroed timestamps.........: 3 (warning: this prevents EAPOL time calculation)
BEACON.................................: 1
EAPOL messages (total).................: 2
EAPOLTIME (measured maximum usec)......: 9999998
EAPOL M1 messages......................: 1
EAPOL M2 messages......................: 1
EAPOL pairs............................: 1
EAPOL pairs written to combi hash file.: 1
EAPOL M12E2............................: 1

$ hcxhashtool -i test.22000 --info=stdout --psk='hashcat!'
SSID......: hashcat.net
MAC_AP....: b0487ad676e2 (TP-LINK TECHNOLOGIES CO.,LTD.)
MAC_CLIENT: 0025cf2db489 (Nokia Danmark A/S)
MP M1M2 E2: not authorized
RC INFO...: replycount checked
MIC.......: d9f3b5b6f744c662518458ac6cc79f11
HASHLINE..: WPA*02*d9f3b5b6f744c662518458ac6cc79f11*b0487ad676e2*0025cf2db489*686173686361742e6e6574*2f0f764c6632d5579c57c3a9fe067a845e22d6435941c1843845db34a2f80dde*0103007502010a0000000000000000000170003e0ad11bc0a9e48679459ebcbffd7ee75697628c371365d7a05e1b35d7d8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020000*00

c1dac5e35b622ee03931e8e12486df3e6a7c04cddff86b8ac72ab16b918649c3*686173686361742e6e6574:hashcat!

OUI information file...: /home/zerobeat/.hcxtools/oui.txt
OUI entires............: 27383
total lines read.......: 1
valid hash lines.......: 1
EAPOL hash lines.......: 1
EAPOL written..........: 1
zhovner commented 4 years ago

hcxhashtool --info is still not working on macOS. I use latest HEAD-4370ffb. I've try both stdout and file path output.

$ cat /tmp/myhashes2.txt
WPA*02*97b6dee2fbd7e1d47ab252c32214d3b8*10feed9bd562*00cdfe10558a*6d6f6c6f6b6f*afb9fe8fc927f72771da8880438fff039ad6eaf7f2a8a90c8cd054244f16e2be*0203007502010a00100000000000000001df3c061e18793c35f0d6794b1ad0d6875a1c2f1abc6be35a72946b59ad6694c9000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020c00*00
WPA*02*c581aae8afb30211441d555c2543a3c2*10feed9bd562*60128b2b6918*6d6f6c6f6b6f*fb56df5d5a9377c722161a4be4490439b5cf96a2b36e416832a9643cfab6ca65*0103007702010a00000000000000000001f8c4429206be88ab23c7a04962d8bfe7a283b58bd922bb7596b47e93fe78762c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001830160100000fac040100000fac040100000fac0200000000*00
WPA*02*5c6faa6a8e3e5ceaf379ef2c395af7e7*10feed9bd562*647033aaf682*6d6f6c6f6b6f*d53323edf1c5e32a8cded1ec54a241860f9162aed5258edf55b3ccff95b9ac53*0203007502010a001000000000000000014cbac3e9f5e5aa1472c89c4515c89a09c2947a9a319e0e624ed1473e4a2c1838000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020c00*00
WPA*02*85ef1323a78904e01f29edf927ed50d8*10feed9bd562*78abbb898296*6d6f6c6f6b6f*9df13312fdf2e37952e9dbea0db72ddd78a4d11e28eba902c575044312362810*0103007502010a000000000000000000015537698ebfe1bf713bcc5240bb8acd3e5baf6ecc4e4f6c0b6796f7fd62a04d68000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020000*00
WPA*01*78837cb042dcb37018d54a3ecfae57d0*3c9872726c1a*9801a78e67f9*4d4754535f47504f4e5f36303037***

$ ./HEAD-4370ffb/hcxhashtool -i /tmp/myhashes2.txt --info=stdout

OUI information file...: /Users/a/.hcxtools/oui.txt
OUI entires............: 27381

$ ./HEAD-4370ffb/hcxhashtool -i /tmp/myhashes2.txt --info=file.txt
$ ls: file.txt: No such file or directory

No. The Linux philosophy (an mine, too) is: Write programs that do one thing and do it well.

Okay, I will use my python script for this.

ZerBea commented 4 years ago

This is a different issue and we have to check it, too. I think it is related to macOS handling of stdout. So, adding that to hcxpcapngtool, will leave us to the same result, because Linux stdout isn't accepted by macOS. If you like, we can use the VM again to hunt for that problem.

zhovner commented 4 years ago

Is there any chance to add simple script like this hashlist.sh to usefulscripts folder before 6.0 release? Maybe with different name. The goal is to get 22000 hashes in stdout just by run one command against pcap dump. I want to refer this script in tutorials.

ZerBea commented 4 years ago

No, because it is too simple (starting only 2 commands). But you can add it to the wiki: https://github.com/ZerBea/hcxtools/wiki or to this thread: https://hashcat.net/forum/thread-6661.html

ZerBea commented 4 years ago

BTW: You haven't open comments/issues on https://github.com/zhovner/hcxtools/blob/master/usefulscripts/hashlist.sh so I write it here. I recommend to convert all(!) possible handshakes to hashline 22000: $ hcxpcapngtool --all -o $tmp_hash_file $1 $ hcxhashtool -i $tmp_hash_file --info=stdout Than you can search for the desired handshake to feed hashcat and/or JtR simply by running additional bash commands within your script.

ZerBea commented 4 years ago

Please take a look at this analysis: https://hashcat.net/forum/thread-8910-post-47468.html#pid47468 and you can imagine, why we convert to hashline 22000 in a first step. It makes it really easy to find a good hashline to feed to hashcat.