Feature Request: support pcapng and gz compressed cap files

[ZerBea]

To make the tool complete, please add support for pcapng (Wireshark and tshark default capture format) and support for gz compressed cap, pcap, pcapng files. That would be a huge improvement. Thanks Mike

[s77rt]

@ZerBea I have just added support for gz compressed (already supported) captures I'm looking into pcapng implementation..

[ZerBea]

That will be great for users of hascat 5.x. Next version will move to new hash line format, because this format has many, many advantages (all bash commands/tools are working on the new format).

[ZerBea]

pcapng is easy to handle and offers many features: https://pcapng.github.io/pcapng/

[s77rt]

@ZerBea pcapng has nothing to do with hashcat, right? unless they change the hccapx structure

[RealEnder]

@s77rt in pcapng you can have additional information in frame comments, which can be useful for correct handshake identification and message_pair semantics. The new hashcat mode (22000) will allow for PBKDF2 heavy operation reuse with same ESSIDs between hccapx and pmkid. You've already got hccapx support, so it will be easy to implement it (as per hashcat/hashcat#1816).

[ZerBea]

@s77rt correct. pcapng is a capture format and has nothing to do with hashcat. As @RealEnder told, is has many advantages. hccapx wiil not change, because it is deprecated. The plan is to remove it by next hashcat version $ hashcat --version v5.1.0-1522-g55d52160 $ hashcat --help | grep 2200 22000 | WPA-PBKDF2-PMKID+EAPOL | Network Protocols 22001 | WPA-PMK-PMKID+EAPOL | Network Protocols First tests, running this new format are very impressive. Reuse PBKDF2 over PMKID and EAPOL. That will save GPU time in case of the same ESSID. Unfortunately, many users running old formats. There are still many very old hccap in the wildness and I'm sure, when format changed to 2200x, there will be tons of deprecated hccapx, too. So, I'm glad to see your cap2hccapx.py tool, because it will still support hccapx.

[ZerBea]

BTW: There are some more bits in the message pair field. They can be used to control hashcats nonce-error-corrections behavior: 5: LE router detected (set to 1) - nonce-error-corrections only for LE necessary 6: BE router detected (set to 1) - nonce-error-corrections only for BE necessary 7: not replaycount checked (set to 1) - replaycount not checked, nonce-error-corrections definitely necessary That is useful in case of a packet loss during reception. To detect NC, just compare at least 2 ANONCEs from different authentication sequences. Compare the last bytes to detect LE/BE and NC.

Your tool is taking care about this message pair fiield - and that is the reason, I like it and stared it.

[s77rt]

Ok, Added pcapng support (not fully tested) @RealEnder will the current hccapx version benefit in any way from pcapng format? @ZerBea can you explain a little more the new message pair field values?

[RealEnder]

To benefit first you have to use dumper, which adds additional info to the frames. You can take a look at ZerBea's repos https://github.com/ZerBea/hcxdumptool and hcxpcapngtool https://github.com/ZerBea/hcxtools. Message pair, as described in hcxpcapngtool has this flags:

bitmask for message pair field:
0: MP info (https://hashcat.net/wiki/doku.php?id=hccapx)
1: MP info (https://hashcat.net/wiki/doku.php?id=hccapx)
2: MP info (https://hashcat.net/wiki/doku.php?id=hccapx)
3: x (unused)
4: ap-less attack (set to 1) - no nonce-error-corrections necessary
5: LE router detected (set to 1) - nonce-error-corrections only for LE necessary
6: BE router detected (set to 1) - nonce-error-corrections only for BE necessary
7: not replaycount checked (set to 1) - replaycount not checked, nonce-error-corrections definitely necessary

Those are mainly used by the cracker to know if it have to do nonce correction of the anonce and if it has to, haw to interpret the corrected value - as big or little endian. This gives us some more performance in the range of 1%-3%.

[ZerBea]

As @RealEnder explained. The message pair bits are used to control hashcat, by setting a special bit. For example, you can tell hashcat to do only NC on LE routers (set bit 5 and bit 7). Doing that, it will save GPU time and make sure the PSK can be recovered if you have a packet loss during reception. Another benefit a the user can decide, which handshake hashcat should use: -hccapx-message-pair | Num | Load only message pairs from hccapx matching X | --hccapx-message-pair=2 For example if your cap2hccapx.py add message pair information 2, the user can decide that hashcat only use handshakes with this message pair.

Now the advantages of pcapng format. Here we use comment fields to tell the conversion tool (in this case cap2hccapx.py) which parameters are in use while dumping/attacking networks. The conversion tool (cap2hccapx.py) can use this information and convert the desired handshakes.That will improve conversion speed. There are common fields, used by all tools (wireshark/tshark/hcxdumptool) and special fields, used by hcxdumptool. Here is an overview of the hcxdumptool comment fields: pcapng option codes (Section Header Block) ENTERPRISE NUMBER 0x2a, 0xce, 0x46, 0xa1 MAGIC NUMBER 0x2a, 0xce, 0x46, 0xa1, 0x79, 0xa0, 0x72, 0x33, 0x83, 0x37, 0x27, 0xab, 0x59, 0x33, 0xb3, 0x62, 0x45, 0x37, 0x11, 0x47, 0xa7, 0xcf, 0x32, 0x7f, 0x8d, 0x69, 0x80, 0xc0, 0x89, 0x5e, 0x5e, 0x98 OPTIONCODE_MACMYORIG 0xf29a (6 byte) OPTIONCODE_MACMYAP 0xf29b (6 byte) OPTIONCODE_RC 0xf29c (8 byte) OPTIONCODE_ANONCE 0xf29d (32 byte) OPTIONCODE_MACMYSTA 0xf29e (6 byte) OPTIONCODE_SNONCE 0xf29f (32 byte) OPTIONCODE_WEAKCANDIDATE 0xf2a0 (32 byte) OPTIONCODE_GPS 0xf2a1 (max 128 byte)

It looks like that: $ hcxpcapngtool -o /tmp/test.22000 /home/zerobeat/Downloads/neu/cap/201912202139.pcapng reading from 201912202139.pcapng...

summary capture file file name..............................: 201912202139.pcapng version (pcapng).......................: 1.0 operating system.......................: Linux 4.19.88-1-ARCH application............................: hcxdumptool 6.0.0 interface name.........................: wlan0 interface vendor.......................: 74da38 weak candidate.........................: 12345678 MAC ACCESS POINT.......................: 00259d58941a (incremented on every new client) MAC CLIENT.............................: dc701466730f REPLAYCOUNT............................: 65261 ANONCE.................................: b03757879e8b8d68445880e116be038003fd00b6bd7b70b284aba1bb12f72348 SNONCE.................................: d9c9c7dcb7dcaee7217716b684c0283337d9851ec1cae3348a058db77b957826 link layer header type.................: DLT_IEEE802_11_RADIO (127) endianess..............................: little endian

or an example of a tshark capture: summary capture file file name..............................: 20191224_092541001.pcapng version (pcapng).......................: 1.0 operating system.......................: N/A application............................: TShark (Wireshark) 3.2.0 (Git commit e0ed4cfa3d72) interface name.........................: N/A weak candidate.........................: N/A link layer header type.................: DLT_IEEE802_11_RADIO (127)

Wireshark/tshark storing less information than hcxdumptool. As you can see, using pcapng, we get some important and useful information about the capture. Chain: dumper/attacker (pcapng information in comment field) -> conversion tool (cap2hccapx.py information in message pair field) -> hashcat (will use this information to improve speed)

[ZerBea]

BTW: We are sharing this additional information. hashcat, JtR, wpa-sec, wifite2 @ kimocoder and hcxdumptool/hcxtools will use them. I added this feature request, here, to share this information, with you, too, because cap2hccapx.py is a nice and portable tool.

[ZerBea]

Counting the time (EAPOLTIME) between two EAPOL messages is a good idea, too, because you can't rely on the replaycount (RC), alone. Even if the replaycount matches, the access point can release his counters and run a complete new randomized ANONCE. In case of a packet loss during capture time, cap2hccpax.py will not notice that by checking RC, only. Result is an uncrackable hccapx record. You can read more here: https://community.cisco.com/t5/wireless-mobility-documents/eap-timers-on-wireless-lan-controllers/ta-p/3122477

Also it is a good idea to use EAPOL from M2 and/or from M4 (if SNONCE is not zeroed and we have no M2). In that case the hccapx can be converted back to a cap file. A M2 contain an unencrypted RSN-IE or a WPA-IE. Here we have useful and important information, too. So, using the M2 should be the goal.

Please take also a look at the new hashline (2200x - hashcat/JtR).

[s77rt]

Added the new message pair bits values. ap-less attack bit is not used yet, i'm looking into this..

@ZerBea

Counting the time (EAPOLTIME) between two EAPOL messages is a good idea

Will be implemented..

Also added the new export format (EAPOL only, PMKID will be added)

$ python3 cap2hccapx.py -i 1.pcapng -a -x hcwpax
Networks detected: 1

[*] BSSID=94:0b:19:ab:9c:3c ESSID=Malak (Length: 5)
 --> STA=e8:3a:12:a3:0c:3f, Message Pair=0, Replay Counter=1, Authenticated=N
 --> STA=e8:3a:12:a3:0c:3f, Message Pair=2, Replay Counter=1, Authenticated=Y

hcWPAx:
WPA*02*dff6195892e87a6f2e0b75f43964f939*940b19ab9c3c*e83a12a30c3f*4d616c616b*7589cee94c178ca7ffb1388389b54827c363f11a865d04d0bf8ca6e7f930f184*0103007502010a0000000000000000000179391c1db3eea921c799b7701506f1f1d1bfdc611df2138f0019198c0fe023cf000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac020000*00
WPA*02*dff6195892e87a6f2e0b75f43964f939*940b19ab9c3c*e83a12a30c3f*4d616c616b*7589cee94c178ca7ffb1388389b54827c363f11a865d04d0bf8ca6e7f930f184*0103007502010a0000000000000000000179391c1db3eea921c799b7701506f1f1d1bfdc611df2138f0019198c0fe023cf000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac020000*02

[ZerBea]

Amazing. Thanks. Now we have a portable tool to convert captures to hccapx. The purpose of bit 4 is to tell hashcat that the hccapx record came from a hcxdumptool. To set it makes only sense on message pair M12E2, because we're attacking a client and will not get a M3 or send a M4. Different to other tools (e.g. aircrack), hcxdumptool/hcxtools put focus on clients, because most of them are weak (IOT devices), while most of the access points are hardened. So why do wasting time and trying to attack a hardened target. I'll take a look at git API to get your email (I think this one is your email: **@gmail.com) address and send you an example.

[ZerBea]

I sent you a mail. Please let me know if the mail has arrived.

[ZerBea]

BTW: PMKID Usually we can get a PMKID from M1, if AP use this method. On the first connection attempt it can be zeroed. This one is useless for us. Also you can retrieve a PMKID from RSN-IE of a reassociationrequest. It is mandatory to take a look inside the RSN-IE to identify the AKM methode (from AKM suite: we need AK_PSK, AK_PSKSHA256), because not all of them running this algorithm: PMKID = HMAC-SHA1-128(PMK, "PMK Name" | MAC_AP | MAC_STA)

For example: https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1535

[ZerBea]

Tested the last commit against the initial commit and noticed huge improvements. Especially the hcwpax output will make life much easier (post hashes in forum threads, running bash tools to manage them, and more...) in combination with latest hashcat and JtR (when Magnum implemented the new hash line, too). cap2hccapx.py become a really amazing tool, now. Good work!

Now, It is time for hccapx to leave us for good (R.I.P).

[s77rt]

Added PMKID support Added hcxdumptool support (ap-less feature)

$ python3 cap2hccapx.py -i capture.pcapng.gz -x hcwpax --quiet
WPA*02*f2537ff8e03fa4574f4504070f0eed80*7cff4dcf039e*e47dbd381c29*465249545a21426f782037343330204c57*74b83979fdfd5a4b218a7cc9449709faf130a26da09a1280076af93324acdc0d*0103007502010a000000000000000000019308f84c66b01981b7191f8a641c2b588a724979a5082877764fcd3f7e4964f1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020000*02
WPA*02*5024a288bbfe19ac0e391d1e5dd814e5*7cff4dcf039e*e47dbd381c29*465249545a21426f782037343330204c57*74b83979fdfd5a4b218a7cc9449709faf130a26da09a1280076af93324acdc0d*0103007502010a00000000000000000001e89c4414c72193ac8acb3abc693cabab21fdfb8b1b0e93d333173fcf887df7cd000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020000*02
WPA*01*a7e73f066dd93e2a5924fcb525fe7b54*708a097c7dd4*b025aa5b15f0*436f7573696e732042617262657273686f70***
WPA*02*14f84d393688e2f5f9f28cc1173b547e*e06066377e4a*804e70199d5a*45617379426f782d383135323538*083d9b303f66f936ea3d01b6e26523d3cd860820ca72c5e5b2a2fe6fbecc969c*0103007502010a0000000000000000f2bf66c63bba715961ec415e38989ba634270f807d0007a06b1e4bd460c8b0ebcf29000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac022c24*82
WPA*02*15b3f5671c4437df45bb31706a4357ae*e06066377e4a*804e70199d5a*45617379426f782d383135323538*083d9b303f66f936ea3d01b6e26523d3cd860820ca72c5e5b2a2fe6fbecc969c*0103007502010a0000000000000000000066c63bba715961ec415e38989ba634270f807d0007a06b1e4bd460c8b0ebcf29000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac022c24*02
WPA*01*18e3300062bff89a52e0f1e188994fc9*02264d8c2d5e*b025aa5b15f0*4d4e4831***
WPA*01*91df949a465b0abe75bbcccae542c931*cca22308d94d*b025aa5b15f0*574c414e2d473450534432***
WPA*02*2b0e589b86b56963edba443322bbcc5f*5c49797346ce*18602431be7a*465249545a21426f782037343930*7039555d663cd2b11d31416df0fee46ab3ba567e40429391aca625d0d8cd06a7*0203007502010a00000000000000000001ee7f0b5edb041b09e27520e79aeb1f121c8a25199665a54f8d0df25c3546f671000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020c00*02
WPA*01*c0304e1fe08f6645380ffca70fd8d995*880355dc429d*b025aa5b15f0*6f322d574c414e3637***
WPA*01*0eb80a564066b8b78d0586dd922681db*04bf6db22d92*b025aa5b15f0*6f322d574c414e3334***
WPA*02*24fa87519f4b1d4751f7ca13c3bedd5d*989bcb7d1419*b8763ffa08cf*4162536f4c7554*7979462be7e86d5c371bcf62813b7b0526d2ce25156825c199bc1aef0b21a5a9*0103007502010a000000000000000000017f371c4676148c509c874cadef66948f8fa224eb0e87f6a8e68c0ec6976e3a32000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac023c00*02
WPA*02*80b884cc6eb8931fb8f0c1ecb7e33daf*3810d543a1fa*3035ad8db504*536b756c6c32303030*241336a80789ee81e66cbd989bfece8661c16397ba95fe862918c091214b39b4*0203007502010a001000000000000000019487ade090f8bc1de0eff8821df677d9becffb3d261667a4616d0b6b2dae928e000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020c00*02

[ZerBea]

Well done - you're prepared for hashcat release 6.0.0, now and you will be prepared for next JtR, too. I also thank you for pointing out some ugly issues within hcxtools/hcxdumptool.