hcxtools 6.3.1 regression test failure (format WPA*01*: additional PMKID MESSAGEPAIR field)

[chenrui333]

👋 while trying to upgrade hcxtools formula to the latest release, I found that some behavior change, which is below

==> diff /private/tmp/hcxtools-test-20230701-26863-19a7yrn/new.22000 /private/tmp/hcxtools-test-20230701-26863-19a7yrn/test.22000
1c1
< WPA*01*4d4fe7aac3a2cecab195321ceb99a7d0*fc690c158264*f4747f87f9f4*686173686361742d6573736964***01
---
> WPA*01*4d4fe7aac3a2cecab195321ceb99a7d0*fc690c158264*f4747f87f9f4*686173686361742d6573736964***

I dont quite understand 01 in `new.22000 output.

If you want to see the full log, it would be in here https://github.com/Homebrew/homebrew-core/actions/runs/5422502608/jobs/9859283379?pr=135390 relates to https://github.com/Homebrew/homebrew-core/pull/135390

[chenrui333]

This is the full test log in my local

test log

``` ==> Testing hcxtools /opt/homebrew/Library/Homebrew/test.rb (Formulary::FromPathLoader): loading /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/hcxtools.rb ==> /opt/homebrew/Cellar/hcxtools/6.3.1/bin/hcxhash2cap --pmkid-eapol=/private/tmp/hcxtools-test-20230701-26863-19a7yrn/test.22000 -c /private/tmp/hcxtools-test-20230701-26863-19a7yrn/test.cap PMKIDs/EAPOL messages written to capfile(s): 1 (0 skipped) ==> /opt/homebrew/Cellar/hcxtools/6.3.1/bin/hcxpcapngtool -o /private/tmp/hcxtools-test-20230701-26863-19a7yrn/new.22000 /private/tmp/hcxtools-test-20230701-26863-19a7yrn/test.cap test.cap 6.3.1 reading from test.cap... summary capture file -------------------- file name................................: test.cap version (pcap/cap).......................: 2.4 (very basic format without any additional information) timestamp minimum (GMT)..................: 01.07.2023 11:27:44 timestamp maximum (GMT)..................: 01.07.2023 11:27:44 used capture interfaces..................: 1 link layer header type...................: DLT_IEEE802_11 (105) very basic format without any additional information about the quality endianness (capture system)..............: little endian packets inside...........................: 2 ESSID (total unique).....................: 1 BEACON (total)...........................: 1 BEACON on 2.4 GHz channel (from IE_TAG)..: 3 BEACON (hcxhash2cap).....................: 1 EAPOL messages (total)...................: 1 EAPOL RSN messages.......................: 1 EAPOL ANONCE error corrections (NC)......: not detected EAPOL M1 messages (total)................: 1 RSN PMKID (total)........................: 1 RSN PMKID (best).........................: 1 RSN PMKID written to 22000 hash file.....: 1 Information: limited dump file format detected! This file format is a very basic format to save captured network data. It is recommended to use PCAP Next Generation dump file format (or pcapng for short) instead. The PCAP Next Generation dump file format is an attempt to overcome the limitations of the currently widely used (but very limited) libpcap (cap, pcap) format. https://www.wireshark.org/docs/wsug_html_chunked/AppFiles.html#ChAppFilesCaptureFilesSection https://github.com/pcapng/pcapng Information: radiotap header is missing! Radiotap is a de facto standard for 802.11 frame injection and reception. The radiotap header format is a mechanism to supply additional information about frames, rom the driver to userspace applications. https://www.radiotap.org/ Information: missing frames! This dump file does not contain undirected proberequest frames. An undirected proberequest may contain information about the PSK. It always happens if the capture file was cleaned or it could happen if filter options are used during capturing. That makes it hard to recover the PSK. Information: missing frames! This dump file does not contain important frames like authentication, association or reassociation. It always happens if the capture file was cleaned or it could happen if filter options are used during capturing. That makes it hard to recover the PSK. Information: missing frames! This dump file does not contain enough EAPOL M1 frames. It always happens if the capture file was cleaned or it could happen if filter options are used during capturing. That makes it impossible to calculate nonce-error-correction values. Information: missing EAPOL M3 frames! This dump file does not contain EAPOL M3 frames (possible packet loss). It strongly recommended to recapture the traffic or to use --all option to convert all possible EAPOL MESSAGE PAIRs. session summary --------------- processed cap files...................: 1 ==> diff /private/tmp/hcxtools-test-20230701-26863-19a7yrn/new.22000 /private/tmp/hcxtools-test-20230701-26863-19a7yrn/test.22000 1c1 < WPA*01*4d4fe7aac3a2cecab195321ceb99a7d0*fc690c158264*f4747f87f9f4*686173686361742d6573736964***01 --- > WPA*01*4d4fe7aac3a2cecab195321ceb99a7d0*fc690c158264*f4747f87f9f4*686173686361742d6573736964*** ```

[ZerBea]

The new (PMKID) MESSAGE PAIR field is explained in changelog:

10.05.2023
==========
hcxpcapngtool: added new option store PMKIDs coming from a CLIENT to a separate hash file.
--pmkid-client=<file>              : output WPA-(MESH/REPEATER)-PMKID hash file (hashcat -m 22000)
to sucessfully recover the PSK from this PMKIDs, it is mandatory to store all PMKIDs coming from a CLIENT to this file

added information about source to the end of WPA*01 hash line:
PMKID from ACCESS POINT: WPA*01*PMKID*MAC_AP*MAC_CLIENT*ESSID***01
PMKID from CLIENT      : WPA*01*PMKID*MAC_AP*MAC_CLIENT*ESSID***10

The new (PMKID) MESSAGE PAIR field is explained in --help, too:

bitmask of PMKID hash line (WPA*01) message pair field:
0: reserved
1: PMKID taken from AP
2: reserved
4: PMKID taken from CLIENT (wlan.da: possible MESH or REPEATER)
5: reserved
6: reserved
7: reserved

hashcat accept the old format and the new format:

$ hashcat -m 22000 "WPA*01*4d4fe7aac3a2cecab195321ceb99a7d0*fc690c158264*f4747f87f9f4*686173686361742d6573736964***01" -a 3 "hashcat!"
hashcat (v6.2.6-611-ga0d509b11) starting
...
Session..........: hashcat
Status...........: Cracked
Hash.Mode........: 22000 (WPA-PBKDF2-PMKID+EAPOL)
Hash.Target......: 4d4fe7aac3a2cecab195321ceb99a7d0:fc690c158264:f4747...-essid
Time.Started.....: Sat Jul  1 18:24:47 2023 (0 secs)
Time.Estimated...: Sat Jul  1 18:24:47 2023 (0 secs)
Kernel.Feature...: Pure Kernel
Guess.Mask.......: hashcat! [8]
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:       78 H/s (0.23ms) @ Accel:8 Loops:128 Thr:512 Vec:1
Recovered........: 1/1 (100.00%) Digests (total), 1/1 (100.00%) Digests (new)
Progress.........: 1/1 (100.00%)
Rejected.........: 0/1 (0.00%)
Restore.Point....: 0/1 (0.00%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidate.Engine.: Device Generator
Candidates.#1....: hashcat! -> hashcat!
Hardware.Mon.#1..: Temp: 35c Fan:  0% Util: 40% Core:2520MHz Mem:10802MHz Bus:16

Started: Sat Jul  1 18:24:46 2023
Stopped: Sat Jul  1 18:24:48 2023

This behavior is wanted (and not a regression), because we need to distinguish if a PMKID was converted from an AP or from a CLIENT.

Please notice: If the PSK is changed, the PSK calculated from the PMKID of the AP (it use the new PSK) can be different to the PSK calculated from the PMKID of the CLIENT (if it still use the old PSK). The user should know to which PMKID the recovered PSK belong. The same applies to some MESH/REPEATERs.

Closed, because this behavior is not a regression (and explained in changelog and --help), but you can still ask your questions here.

[chenrui333]

@ZerBea thanks for the quick response!! Appreciate it. :)

[ZerBea]

Running hcxpcapngtool --all can lead at least up to three different PSKs:

EAPOL M1M2 (CHALLENGE) e.g. challenge PSK = 12345678
EAPOL M2M3 (AUTHORIZED) e.g. authorized PSK = passwordnew
PMKID AP (AUTHORIZED) e.g. authorized PSK = passwordnew
PMKID CLIENT (old AUTHORIZED) e.g. old authorized PSK = passwordold

EAPOL MESSAGE PAIR field show an information about the used hash (CHALLENGE or AUTHORIZED). From now on, PMKID MESSAGE PAIR field will show a similar information (PMKID from AP, PMKID from CLIENT), too.

[ZerBea]

I forgot to mention: Since the implementation of hash mode 22000 the MESSAGE PAIR field is a fixed part of the hash line. It is used on WPA02 (EAPOL), but it has never been used on WPA01 (empty field). From now on, it is used on WPA*01, too.

[chenrui333]

thanks for the notes! I am all set now. 👍