search

ZerBea / hcxtools

A small set of tools to convert packets from capture files to hash files for use with Hashcat or John the Ripper.
MIT License
2.01k stars 392 forks source link

Please add more detailed valid hash info #338

Closed LLH-l closed 2 months ago

LLH-l commented 5 months ago

When using hcxpcapngtool -- add - o More detailed information needs to be added, which can be used to filter valid hashes more accurately and increase the recognition of absolutely valid hashes E.G time interval, accord with M1 to M3 authentication sequence conditions

If the packet does not have M3, only M1M2
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00    12.06.2024 15:23:18 261628000   M1M2
If the packet does not have M1, only M2M3
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02    12.06.2024 15:23:18 261628000   M2M3
If the data packet has M1M2M3,  convert M21E2  ( If time interval, authentication sequence conditions, and m1m3 NONCE match )
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00    12.06.2024 15:23:18 261628000   M1M2M3

If the data packet has M1M2M3,  convert M21E2  ( If m1m3 NONCE no match )
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00    12.06.2024 15:23:18 261628000   M1M2

If the data packet has M1M2M3,  convert M32E2  ( If time interval, authentication sequence conditions, and m1m3 NONCE match )
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02    12.06.2024 15:23:18 261628000   M1M2M3

If the data packet has M1M2M3,  convert M32E2  ( If m1m3 NONCE no match )
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02    12.06.2024 15:23:18 261628000   M2M3

In this way, 100% valid hashes can be filtered out through "M1M2M3" This may take your some time to change !

ZerBea commented 5 months ago

partly done by this commit: https://github.com/ZerBea/hcxtools/commit/9a06474ca100faf936c5d21d3c2b014e6a124839

$ wget https://github.com/ZerBea/hcxtools/files/12792860/e.g.zip
$ unzip e.g.zip
$ hcxpcapngtool e.g.pcap -o all.22000 --add-timestamp --all

Now you can use bash tools to e.g. sort handshakes by lowest timegap:

$ cat all.22000 | grep "WPA\*02" | sort -t$'\t' -k3
WPA*02*2f6f8e8b95fcf1537de07424b61238a3*786256a37308*c8ddc9f59185*e68891e79a84e5a4a7e88fa0e8909d*da74e2ab310bf196035a7567f9d66838e404fc1b1d1c4d62172a4c7335c265ac*0103007502010a00000000000000000000f094e260051da6561e9c789e32045bd3877ad34adeadab2100dc8c4bf7be4cac000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020000*02 04.11.2019 16:36:50 7184000 M32E2
WPA*02*2f8582bf98703663056b7ef4ba991d4d*786256a37308*c8ddc9f59185*e68891e79a84e5a4a7e88fa0e8909d*f094e260051da6561e9c789e32045bd3877ad34adeadab2100dc8c4bf7be4cac*010300970213ca00100000000000000001da74e2ab310bf196035a7567f9d66838e404fc1b1d1c4d62172a4c7335c265acda74e2ab310bf196035a7567f9d66838dea800000000000000000000000000000000000000000000000000000000000000388ccda4d44ac706c27275c61527309bbb46194bab7deddc3f317b7d4a1e5a133599f9ea4a2bb5ac9d025f2f8fe6b2b665b22a2e8a9b1052d0*13 04.11.2019 16:36:50 7184000 M32E3
WPA*02*2f6f8e8b95fcf1537de07424b61238a3*786256a37308*c8ddc9f59185*e68891e79a84e5a4a7e88fa0e8909d*da74e2ab310bf196035a7567f9d66838e404fc1b1d1c4d62172a4c7335c265ac*0103007502010a00000000000000000000f094e260051da6561e9c789e32045bd3877ad34adeadab2100dc8c4bf7be4cac000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac020000*02 04.11.2019 16:36:50 9232000 M32E2
WPA*02*2f8582bf98703663056b7ef4ba991d4d*786256a37308*c8ddc9f59185*e68891e79a84e5a4a7e88fa0e8909d*f094e260051da6561e9c789e32045bd3877ad34adeadab2100dc8c4bf7be4cac*010300970213ca00100000000000000001da74e2ab310bf196035a7567f9d66838e404fc1b1d1c4d62172a4c7335c265acda74e2ab310bf196035a7567f9d66838dea800000000000000000000000000000000000000000000000000000000000000388ccda4d44ac706c27275c61527309bbb46194bab7deddc3f317b7d4a1e5a133599f9ea4a2bb5ac9d025f2f8fe6b2b665b22a2e8a9b1052d0*13 04.11.2019 16:36:50 9232000 M32E3

In this way, 100% valid hashes can be filtered out through "M1M2M3" This is not feasible because hcxtools, hashcat and JtR only use MESSAGE PAIRs: Due to performance reasons none of this tools run an additional stage to count MESSAGE TRIPLETS.

ZerBea commented 5 months ago

This important to know: In a hc22000 file, an EAPOL M12E2 and an EAPOL M32E2 are 100% identical. The only difference between them is: EAPOL M12E2 == CHALLENGE EAPOL M32E2 == AUTHORIZATION

BTW: By the latest commit, a hc22000 now contain redundant information. I actually wanted to avoid that! M32E3 is absolutely the same as the MESSAGPAIR FIELD: "...*x3" Due to this additional (redundant) field, the hash files grow and bash tools must handle an additional field.

A hc22000 hash file is a machine-readable file. It is not intended that people evaluate the data by hand.

LLH-l commented 5 months ago

partly done by this commit: 9a06474

WPA022f6f8e8b95fcf1537de07424b61238a3786256a37308c8ddc9f59185e68891e79a84e5a4a7e88fa0e8909dda74e2ab310bf196035a7567f9d66838e404fc1b1d1c4d62172a4c7335c265ac0103007502010a00000000000000000000f094e260051da6561e9c789e32045bd3877ad34adeadab2100dc8c4bf7be4cac000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac040100000fac040100000fac02000002 04.11.2019 16:36:50 9232000 M32E2 WPA022f8582bf98703663056b7ef4ba991d4d786256a37308c8ddc9f59185e68891e79a84e5a4a7e88fa0e8909df094e260051da6561e9c789e32045bd3877ad34adeadab2100dc8c4bf7be4cac010300970213ca00100000000000000001da74e2ab310bf196035a7567f9d66838e404fc1b1d1c4d62172a4c7335c265acda74e2ab310bf196035a7567f9d66838dea800000000000000000000000000000000000000000000000000000000000000388ccda4d44ac706c27275c61527309bbb46194bab7deddc3f317b7d4a1e5a133599f9ea4a2bb5ac9d025f2f8fe6b2b665b22a2e8a9b1052d013 04.11.2019 16:36:50 9232000 M32E3

Not right , add info like "M32E2" is redundana

WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00    12.06.2024 15:23:18 261628000   M12E2
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02    12.06.2024 15:23:18 261628000   M32E2

Because from the bitmask, can be inferred which messages it is composed

WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00==M12E2
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02==M32E2

So, best thing is me propose improvements above Count valid MESSAGE TRIPLETS

ZerBea commented 5 months ago

Ok, I remove this field.

LLH-l commented 3 months ago

If no, it will encounter problems in the following situations M1M2 PSK is false, or may be unknown PSK

453 Oct 27, 2021 23:34:21.067406000     1   0   35a536ee5331872415e96b2f0b25cb2dc4bca645865a7e06e14bb45869344bb4    38:53:9c:64:02:e6   08:40:f3:f8:95:c8
456 Oct 27, 2021 23:34:21.067474000     2   0   d3c955eecd12595d553f5486cb83fc8e48a9fd01deb821a6de647db8d1f05f7b    08:40:f3:f8:95:c8   38:53:9c:64:02:e6

2453    Oct 27, 2021 23:34:28.167406000     1   0   9b7b85edc4b433bf16264a6baa796eb8806704d0a0aa1e197911fe401422c432    38:53:9c:64:02:e6   08:40:f3:f8:95:c8
2456    Oct 27, 2021 23:34:28.170474000     2   0   a336d050114c239c14a701013231ca8d9c25e9d4465801016a2235c5789dcf96    08:40:f3:f8:95:c8   38:53:9c:64:02:e6
2458    Oct 27, 2021 23:34:28.184576000     3   1   9b7b85edc4b433bf16264a6baa796eb8806704d0a0aa1e197911fe401422c432    38:53:9c:64:02:e6   08:40:f3:f8:95:c8

etc.this situation

For this situation, your may say, use --all, but use --all (not checking zeroed PMK, NC, etc.) Will convert more hashes So, we need optimize of useing - o (equivalent to implementing title mentioned above) When using - o, priority should be given to converting valid message pairs where M1 and M3 NONCE are same

ZerBea commented 3 months ago

No, because hashcat can do this better (due to its nonce-error-corrections) and much faster (due to using GPU). Ignoring hashes calculated from a zeroed PMKID is an optimization exclusive for default mode (get best hash). It is running on CPU only and that is extremely slow. Running this test on all MESSAGE PAIR combinations and all PMKIDs will take a several of hours. It is much faster to use GPU for this task. Hashcat hashcat mode -m 22001 in combination with --nonce-error-corrections and --remove is designed to clean (remove known hashes from known PMKs, remove hashes calculated from zeroed PMKs, ...) a hash file converted by hcxpcapngtool.

Example to remove hashes (PMKID and EAPOL):

$ echo "0000000000000000000000000000000000000000000000000000000000000000" > zeroedpmk.list
$ hashcat -m 220001 --nonce-error-corrections=16 --remove hashfile.hc22000 zeroedpmk.list

This kind of cleaning can be done on all hash files using all PMKs (e.g. from hashcat pot file).

Get PMKs from hashcat 2200x potfile:

$ cut -c -64 hashcat.potfile > pmk.list
$ hashcat -m 22001 --nonce-error-corrections=16 --remove hashfile.hc22000 pmk.list

You can always combine this 2 procedures:

$ echo "0000000000000000000000000000000000000000000000000000000000000000" > pmk.list
$ cut -c -64 hashcat.potfile >> pmk.list
$ hashcat -m 22001 --nonce-error-corrections=16 --remove hashfile.hc22000 pmk.list

nonce-error-corrections=16 is only an example, depending on the quality for your dump file you can use much lower or much higher values.

BTW: hashmode 22001 is really fast: Speed.#1.........: 208.8 MH/s (0.00ms) @ Accel:128 Loops:1024 Thr:512 Vec:1 compared to hash mode 22000 Speed.#1.........: 914.5 kH/s (1.55ms) @ Accel:8 Loops:256 Thr:256 Vec:1

I suggest to read everything about hash mode 22001 on hash cat forum. I recommend to use hash mode 22001 to clean your hash files.

LLH-l commented 3 months ago

Although it can handle some garbage hashes, and solve some problems But when using -o, the above-mentioned issue still will occur

ZerBea commented 3 months ago

Please add an example dump file.

ZerBea commented 3 months ago

Important notice: At time of converting a MESSAGE PAIR, hcxpcapngtool does not know if the PSK is valid or not, because there is absolutely no "cracking capability". Even if M1 ANONCE == M3 ANONCE it does not make sure that the M2 really belongs to the same AUTHENTICATION sequence.

From your example:

453 Oct 27, 2021 23:34:21.067406000     1   0   35a536ee5331872415e96b2f0b25cb2dc4bca645865a7e06e14bb45869344bb4    38:53:9c:64:02:e6   08:40:f3:f8:95:c8
456 Oct 27, 2021 23:34:21.067474000     2   0   d3c955eecd12595d553f5486cb83fc8e48a9fd01deb821a6de647db8d1f05f7b    08:40:f3:f8:95:c8   38:53:9c:64:02:e6

2453    Oct 27, 2021 23:34:28.167406000     1   0   9b7b85edc4b433bf16264a6baa796eb8806704d0a0aa1e197911fe401422c432    38:53:9c:64:02:e6   08:40:f3:f8:95:c8
2456    Oct 27, 2021 23:34:28.170474000     2   0   a336d050114c239c14a701013231ca8d9c25e9d4465801016a2235c5789dcf96    08:40:f3:f8:95:c8   38:53:9c:64:02:e6
2458    Oct 27, 2021 23:34:28.184576000     3   1   9b7b85edc4b433bf16264a6baa796eb8806704d0a0aa1e197911fe401422c432    38:53:9c:64:02:e6   08:40:f3:f8:95:c8

Both are valid (time between MESSAGES is ok, REPLAYCOUNT is ok, MESSAGE NUMBER is in sequence). How did you find out which one of them is crackable and which one not?

Please don't forget: Do not AUTHENTICATE / ASSOCIATE to the target, while an attack is running! Make absolutely sure that the dump file is not crappy. Hcxpcapngtool is the standard conversion tool for hcxdumptool/hcxlabtool pcapng files! If you try to convert crappy dump files recorded by passive dumper tools, hcxpcapngtool shows a warning or an error. If you convert it anyway despite all the warnings you can't expect that the automatic mode is working. If you need more information about the content of the dump file it is mandatory to analyze it. A hash file (hc22000) is not designed to be human readable! An analyses of a hc22000 is useless, because hc22000 hash files do not contain all information.

LLH-l commented 3 months ago

Even if M1 ANONCE == M3 ANONCE it does not make sure that the M2 really belongs to the same AUTHENTICATION sequence.

2453    Oct 27, 2021 23:34:28.167406000     1   0   9b7b85edc4b433bf16264a6baa796eb8806704d0a0aa1e197911fe401422c432    38:53:9c:64:02:e6   08:40:f3:f8:95:c8
2456    Oct 27, 2021 23:34:28.170474000     2   0   0336d050114c239c14a701013231ca8d9c25e9d4465801016a2235c5789dcf96    08:40:f3:f8:95:c8   38:53:9c:64:02:e6
2459    Oct 27, 2021 23:34:28.170474000     2   0   b336d050114c239c14a701013231ca8d9c25e9d4465801016a2235c5789dcf96    08:40:f3:f8:95:c8   38:53:9c:64:02:e6
2465    Oct 27, 2021 23:34:28.170474000     2   0   4336d050114c239c14a701013231ca8d9c25e9d4465801016a2235c5789dcf96    08:40:f3:f8:95:c8   38:53:9c:64:02:e6
2470    Oct 27, 2021 23:34:28.184576000     3   1   9b7b85edc4b433bf16264a6baa796eb8806704d0a0aa1e197911fe401422c432    38:53:9c:64:02:e6   08:40:f3:f8:95:c8

Well, I agree your point of view, even if M1==M3 NONCE， If there are multiple M2 in the middle, and if the MIC is different, It Still unable to solve。 Must use all ！ This issue closed

LLH-l commented 3 months ago

Halo, No We should not discuss using - o or - all We seem deviated from the topic  What we are discussing is: Add more detailed valid hash info This issue still need opened

ZerBea commented 3 months ago

I still don't understand the problem.

hcxpcapngtool is a tool that convert hashes from a dump file to a hc22000
hcxhashtool is a tool that provide information about the hc22000 file

An example from here: https://wiki.wireshark.org/SampleCaptures: $ wget https://wiki.wireshark.org/uploads/__moin_import__/attachments/SampleCaptures/wpa-Induction.pcap

Get general information about the dump file:

$  hcxpcapngtool wpa-Induction.pcap
hcxpcapngtool 6.3.4-46-g5377fda reading from wpa-Induction.pcap...

summary capture file
--------------------
file name................................: wpa-Induction.pcap
version (pcap/cap).......................: 2.4 (very basic format without any additional information)
timestamp minimum (timestamp)............: 04.01.2007 06:14:45 (1167891285)
timestamp maximum (timestamp)............: 04.01.2007 06:15:26 (1167891326)
duration of the dump tool (seconds)......: 40
used capture interfaces..................: 1
link layer header type...................: DLT_IEEE802_11_RADIO (127)
endianness (capture system)..............: little endian
packets inside...........................: 1093
frames with correct FCS..................: 1080
packets received on 2.4 GHz..............: 1093
WIRELESS DISTRIBUTION SYSTEM.............: 1
ESSID (total unique).....................: 2
BEACON (total)...........................: 398
BEACON on 2.4 GHz channel (from IE_TAG)..: 1 
PROBEREQUEST (undirected)................: 12
PROBEREQUEST (directed)..................: 1
PROBERESPONSE (total)....................: 26
DISASSOCIATION (total)...................: 1
AUTHENTICATION (total)...................: 2
AUTHENTICATION (OPEN SYSTEM).............: 2
ASSOCIATIONREQUEST (total)...............: 1
ASSOCIATIONREQUEST (PSK).................: 1
RESERVED MANAGEMENT frame................: 4
WPA encrypted............................: 280
EAPOL messages (total)...................: 4
EAPOL RSN messages.......................: 4
EAPOLTIME gap (measured maximum msec)....: 4
EAPOL ANONCE error corrections (NC)......: not detected
EAPOL M1 messages (total)................: 1
EAPOL M2 messages (total)................: 1
EAPOL M3 messages (total)................: 1
EAPOL M4 messages (total)................: 1
EAPOL M4 messages (zeroed NONCE).........: 1
EAPOL pairs (total)......................: 2
EAPOL pairs (best).......................: 1
EAPOL M12E2 (challenge)..................: 1
RSN PMKID (total)........................: 1
RSN PMKID (from zeroed PMK)..............: 1 (not converted by default options - use --all)

frequency statistics from radiotap header (frequency: received packets)
-----------------------------------------------------------------------
 2412: 1093 

Information: limited dump file format detected!
This file format is a very basic format to save captured network data.
It is recommended to use PCAP Next Generation dump file format (or pcapng for short) instead. The PCAP Next Generation dump file format is an attempt to overcome the limitations of the currently widely used (but very limited) libpcap (cap, pcap) format.
https://www.wireshark.org/docs/wsug_html_chunked/AppFiles.html#ChAppFilesCaptureFilesSection
https://github.com/pcapng/pcapng

Information: missing frames!
This dump file does not contain enough EAPOL M1 frames.
It always happens if the capture file was cleaned or it could happen if filter options are used during capturing.
That makes it impossible to calculate nonce-error-correction values.
Duration of the dump tool was a way too short to capture enough additional information.

Information: no hashes written to hash files

session summary
---------------
processed cap files...................: 1

We got some warnings, so it is a good idea to convert all hashes:

$  hcxpcapngtool -o test.hc22000 --all wpa-Induction.pcap
hcxpcapngtool 6.3.4-46-g5377fda reading from wpa-Induction.pcap...

summary capture file
--------------------
file name................................: wpa-Induction.pcap
version (pcap/cap).......................: 2.4 (very basic format without any additional information)
timestamp minimum (timestamp)............: 04.01.2007 06:14:45 (1167891285)
timestamp maximum (timestamp)............: 04.01.2007 06:15:26 (1167891326)
duration of the dump tool (seconds)......: 40
used capture interfaces..................: 1
link layer header type...................: DLT_IEEE802_11_RADIO (127)
endianness (capture system)..............: little endian
packets inside...........................: 1093
frames with correct FCS..................: 1080
packets received on 2.4 GHz..............: 1093
WIRELESS DISTRIBUTION SYSTEM.............: 1
ESSID (total unique).....................: 2
BEACON (total)...........................: 398
BEACON on 2.4 GHz channel (from IE_TAG)..: 1 
PROBEREQUEST (undirected)................: 12
PROBEREQUEST (directed)..................: 1
PROBERESPONSE (total)....................: 26
DISASSOCIATION (total)...................: 1
AUTHENTICATION (total)...................: 2
AUTHENTICATION (OPEN SYSTEM).............: 2
ASSOCIATIONREQUEST (total)...............: 1
ASSOCIATIONREQUEST (PSK).................: 1
RESERVED MANAGEMENT frame................: 4
WPA encrypted............................: 280
EAPOL messages (total)...................: 4
EAPOL RSN messages.......................: 4
EAPOLTIME gap (measured maximum msec)....: 4
EAPOL ANONCE error corrections (NC)......: not detected
EAPOL M1 messages (total)................: 1
EAPOL M2 messages (total)................: 1
EAPOL M3 messages (total)................: 1
EAPOL M4 messages (total)................: 1
EAPOL M4 messages (zeroed NONCE).........: 1
EAPOL pairs (total)......................: 3
EAPOL pairs (useful).....................: 3
EAPOL pairs written to 22000 hash file...: 3 (RC checked)
EAPOL M12E2 (challenge)..................: 1
EAPOL M32E2 (authorized).................: 1
EAPOL M32E3 (authorized).................: 1
RSN PMKID (total)........................: 1
RSN PMKID (from zeroed PMK)..............: 1 (not converted by default options - use --all)
RSN PMKID (useful).......................: 1
RSN PMKID written to 22000 hash file.....: 1

frequency statistics from radiotap header (frequency: received packets)
-----------------------------------------------------------------------
 2412: 1093 

Information: limited dump file format detected!
This file format is a very basic format to save captured network data.
It is recommended to use PCAP Next Generation dump file format (or pcapng for short) instead. The PCAP Next Generation dump file format is an attempt to overcome the limitations of the currently widely used (but very limited) libpcap (cap, pcap) format.
https://www.wireshark.org/docs/wsug_html_chunked/AppFiles.html#ChAppFilesCaptureFilesSection
https://github.com/pcapng/pcapng

Information: missing frames!
This dump file does not contain enough EAPOL M1 frames.
It always happens if the capture file was cleaned or it could happen if filter options are used during capturing.
That makes it impossible to calculate nonce-error-correction values.
Duration of the dump tool was a way too short to capture enough additional information.

session summary
---------------
processed cap files...................: 1

Get some information about the hash file:

$ hcxhashtool -i test.hc22000 --info=stdout
SSID.......: Coherer
MAC_AP.....: 000c4182b255 (Cisco-Linksys, LLC)
MAC_CLIENT.: 000d9382363a (Apple, Inc.)
PMKID......: 592da88096c461da246c69001e877f3d
HASHLINE...: WPA*01*592da88096c461da246c69001e877f3d*000c4182b255*000d9382363a*436f6865726572***

SSID.......: Coherer
MAC_AP.....: 000c4182b255 (Cisco-Linksys, LLC)
MAC_CLIENT.: 000d9382363a (Apple, Inc.)
VERSION....: 802.1X-2004 (2)
KEY VERSION: WPA2
REPLAYCOUNT: 0
NC INFO....: hashcat default NC activated
EAPOL MSG..: 2
MP M1M2 E2.: challenge
MIC........: a462a7029ad5ba30b6af0df391988e45
HASHLINE...: WPA*02*a462a7029ad5ba30b6af0df391988e45*000c4182b255*000d9382363a*436f6865726572*3e8e967dacd960324cac5b6aa721235bf57b949771c867989f49d04ed47c6933*0203007502010a00100000000000000000cdf405ceb9d889ef3dec42609828fae546b7add7baecbb1a394eac5214b1d386000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac020000*80

SSID.......: Coherer
MAC_AP.....: 000c4182b255 (Cisco-Linksys, LLC)
MAC_CLIENT.: 000d9382363a (Apple, Inc.)
VERSION....: 802.1X-2004 (2)
KEY VERSION: WPA2
REPLAYCOUNT: 0
NC INFO....: hashcat default NC activated
EAPOL MSG..: 2
MP M2M3 E2.: authorized
MIC........: a462a7029ad5ba30b6af0df391988e45
HASHLINE...: WPA*02*a462a7029ad5ba30b6af0df391988e45*000c4182b255*000d9382363a*436f6865726572*3e8e967dacd960324cac5b6aa721235bf57b949771c867989f49d04ed47c6933*0203007502010a00100000000000000000cdf405ceb9d889ef3dec42609828fae546b7add7baecbb1a394eac5214b1d386000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac020000*82

SSID.......: Coherer
MAC_AP.....: 000c4182b255 (Cisco-Linksys, LLC)
MAC_CLIENT.: 000d9382363a (Apple, Inc.)
VERSION....: 802.1X-2004 (2)
KEY VERSION: WPA2
REPLAYCOUNT: 1
NC INFO....: ROGUE attack / NC deactivated
EAPOL MSG..: 3
MP M2M3 E3.: authorized
MIC........: 7d0af6df51e99cde7a187453f0f93537
HASHLINE...: WPA*02*7d0af6df51e99cde7a187453f0f93537*000c4182b255*000d9382363a*436f6865726572*cdf405ceb9d889ef3dec42609828fae546b7add7baecbb1a394eac5214b1d386*020300af0213ca001000000000000000013e8e967dacd960324cac5b6aa721235bf57b949771c867989f49d04ed47c6933f57b949771c867989f49d04ed47c6934cf020000000000000000000000000000000000000000000000000000000000000050cfa72cde35b2c1e2319255806ab364179fd9673041b9a5939fa1a2010d2ac794e25168055f794ddc1fdfae3521f4446bfd11da98345f543df6ce199df8fe48f8cdd17adca87bf45711183c496d41aa0c*13

We don't want to run a challenge message pair (possible false PSK) - so we take an authorized one (M2M3 E2):

SSID.......: Coherer
MAC_AP.....: 000c4182b255 (Cisco-Linksys, LLC)
MAC_CLIENT.: 000d9382363a (Apple, Inc.)
VERSION....: 802.1X-2004 (2)
KEY VERSION: WPA2
REPLAYCOUNT: 0
NC INFO....: hashcat default NC activated
EAPOL MSG..: 2
MP M2M3 E2.: authorized
MIC........: a462a7029ad5ba30b6af0df391988e45
HASHLINE...: WPA*02*a462a7029ad5ba30b6af0df391988e45*000c4182b255*000d9382363a*436f6865726572*3e8e967dacd960324cac5b6aa721235bf57b949771c867989f49d04ed47c6933*0203007502010a00100000000000000000cdf405ceb9d889ef3dec42609828fae546b7add7baecbb1a394eac5214b1d386000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac020000*82

Get the word list: $ wget https://wpa-sec.stanev.org/dict/cracked.txt.gz

Run hashcat:

$ hashcat -m 22000 "WPA*02*a462a7029ad5ba30b6af0df391988e45*000c4182b255*000d9382363a*436f6865726572*3e8e967dacd960324cac5b6aa721235bf57b949771c867989f49d04ed47c6933*0203007502010a00100000000000000000cdf405ceb9d889ef3dec42609828fae546b7add7baecbb1a394eac5214b1d386000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac020000*82" cracked.txt.gz
hashcat (v6.2.6-848-gc1a10518f) starting
...
a462a7029ad5ba30b6af0df391988e45:000c4182b255:000d9382363a:Coherer:Induction

Session..........: hashcat
Status...........: Cracked
Hash.Mode........: 22000 (WPA-PBKDF2-PMKID+EAPOL)
Hash.Target......: a462a7029ad5ba30b6af0df391988e45:000c4182b255:000d9...oherer
Time.Started.....: Sat Aug 17 17:49:39 2024 (0 secs)
Time.Estimated...: Sat Aug 17 17:49:39 2024 (0 secs)
Kernel.Feature...: Pure Kernel
Guess.Base.......: File (cracked.txt.gz)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:   921.2 kH/s (6.25ms) @ Accel:8 Loops:256 Thr:256 Vec:1
Recovered........: 1/1 (100.00%) Digests (total), 1/1 (100.00%) Digests (new)
Progress.........: 311297/609480 (51.08%)
Rejected.........: 1/311297 (0.00%)
Restore.Point....: 155648/609480 (25.54%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:0-1
Candidate.Engine.: Device Generator
Candidates.#1....: 47050488 -> 3209148519
Hardware.Mon.#1..: Temp: 52c Fan:  0% Util: 38% Core:2835MHz Mem:11201MHz Bus:16

Started: Sat Aug 17 17:49:37 2024
Stopped: Sat Aug 17 17:49:40 2024

I don't see a problem: If hcxpcapngtool shows me a few warnings, "--all" is a good decision to convert the dump file. Get all available information of the hashes (hcxhashtool). If you don't want to use the automatic "best hash" mode, manually choose a message pair of which you think it is good. I know you can do this, because here: https://github.com/ZerBea/hcxtools/issues/338#issuecomment-2292669393 You told me the first is bad while the second is good. Run hashcat on it to recover the PSK.

ZerBea commented 3 months ago

I'm sure you notice the difference between a machine readable hc22000 file converted by hcxpcapngtool: WPA*02*a462a7029ad5ba30b6af0df391988e45*000c4182b255*000d9382363a*436f6865726572*3e8e967dacd960324cac5b6aa721235bf57b949771c867989f49d04ed47c6933*0203007502010a00100000000000000000cdf405ceb9d889ef3dec42609828fae546b7add7baecbb1a394eac5214b1d386000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac020000*82

and the human readable output of hcxhashtool:

SSID.......: Coherer
MAC_AP.....: 000c4182b255 (Cisco-Linksys, LLC)
MAC_CLIENT.: 000d9382363a (Apple, Inc.)
VERSION....: 802.1X-2004 (2)
KEY VERSION: WPA2
REPLAYCOUNT: 0
NC INFO....: hashcat default NC activated
EAPOL MSG..: 2
MP M2M3 E2.: authorized
MIC........: a462a7029ad5ba30b6af0df391988e45
HASHLINE...: WPA*02*a462a7029ad5ba30b6af0df391988e45*000c4182b255*000d9382363a*436f6865726572*3e8e967dacd960324cac5b6aa721235bf57b949771c867989f49d04ed47c6933*0203007502010a00100000000000000000cdf405ceb9d889ef3dec42609828fae546b7add7baecbb1a394eac5214b1d386000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001630140100000fac020100000fac040100000fac020000*82

For me it doesn't make sense to add human readable information to a (by hashcat or JtR) machine readable hash file!

ZerBea commented 2 months ago

Same source, but a different approach:

$ wget https://wiki.wireshark.org/uploads/__moin_import__/attachments/SampleCaptures/wpa-Induction.pcap
$ hcxpcapngtool -o test.hc22000 --all wpa-Induction.pcap
$ hcxhashtool -i test.hc22000 --authorized -o auth.hc22000

OUI information file..........: /home/zerobeat/.hcxtools/oui.txt
OUI entries...................: 35167
total lines read..............: 4
valid hash lines..............: 4
PMKID hash lines..............: 1
EAPOL hash lines..............: 3
filter by status..............: authorized (M1M4, M2M3 or M3M4)
EAPOL written.................: 2

$ wget https://wpa-sec.stanev.org/dict/cracked.txt.gz
$ hashcat -m 22000 auth.hc22000 cracked.txt.gz
hashcat (v6.2.6-848-gc1a10518f) starting
...
a462a7029ad5ba30b6af0df391988e45:000c4182b255:000d9382363a:Coherer:Induction
7d0af6df51e99cde7a187453f0f93537:000c4182b255:000d9382363a:Coherer:Induction

Session..........: hashcat
Status...........: Cracked
Hash.Mode........: 22000 (WPA-PBKDF2-PMKID+EAPOL)
Hash.Target......: auth.hc22000
Time.Started.....: Sat Aug 17 19:18:16 2024 (0 secs)
Time.Estimated...: Sat Aug 17 19:18:16 2024 (0 secs)
Kernel.Feature...: Pure Kernel
Guess.Base.......: File (cracked.txt.gz)
Guess.Queue......: 1/1 (100.00%)
Speed.#1.........:   928.0 kH/s (6.20ms) @ Accel:4 Loops:256 Thr:512 Vec:1
Recovered........: 2/2 (100.00%) Digests (total), 2/2 (100.00%) Digests (new)
Progress.........: 311297/609495 (51.07%)
Rejected.........: 1/311297 (0.00%)
Restore.Point....: 155648/609495 (25.54%)
Restore.Sub.#1...: Salt:0 Amplifier:0-1 Iteration:1-3
Candidate.Engine.: Device Generator
Candidates.#1....: 47049121 -> 45073180
Hardware.Mon.#1..: Temp: 49c Fan:  0% Util: 40% Core:2850MHz Mem:11201MHz Bus:16

Started: Sat Aug 17 19:18:14 2024
Stopped: Sat Aug 17 19:18:17 2024

This is the recommended way: hcxdumptool/hcxlabtool -> hcxpcapngtool -> hcxhashtool -> hashcat/JtR Really no one evaluates a machine readable hc22000 by hand! There is absolutely no need to add more information to it.

These are only two examples how to use hcxhashtool. There are much more filter options which can do the work for you, e.g hcxhashtool in combination with bash tools diff, grep and sed:

$ hcxpcapngtool -o test.hc22000 --all wpa-Induction.pcap
$ hcxhashtool -i test.hc22000 --authorized -o auth.hc22000
$ diff test.hc22000 auth.hc22000 | grep '^>' | sed -e 's/> //' > remaining.hc22000

I suggest to code simple bash scripts tailored to your task that include all this this filter options. There is no need to add another output to hc22000 files, because existing hcxtools and bash commands provide all this. Just play around with them!

ZerBea commented 2 months ago

This is a very basic script that converts a dump file into different hc22000 files (authorized, PMKID):

#!/bin/bash

hcxpcapngtool --all $1 -o all.hc22000
hcxhashtool -i all.hc22000 --authorized -o authorized.hc22000
hcxhashtool -i all.hc22000 --type=1 -o pmkid.hc22000

example:

$ ./convertdump wpa-Induction.pcap
hcxpcapngtool 6.3.4-46-g5377fda reading from wpa-Induction.pcap...

summary capture file
--------------------
file name................................: wpa-Induction.pcap
version (pcap/cap).......................: 2.4 (very basic format without any additional information)
timestamp minimum (timestamp)............: 04.01.2007 06:14:45 (1167891285)
timestamp maximum (timestamp)............: 04.01.2007 06:15:26 (1167891326)
duration of the dump tool (seconds)......: 40
used capture interfaces..................: 1
link layer header type...................: DLT_IEEE802_11_RADIO (127)
endianness (capture system)..............: little endian
packets inside...........................: 1093
frames with correct FCS..................: 1080
packets received on 2.4 GHz..............: 1093
WIRELESS DISTRIBUTION SYSTEM.............: 1
ESSID (total unique).....................: 2
BEACON (total)...........................: 398
BEACON on 2.4 GHz channel (from IE_TAG)..: 1 
PROBEREQUEST (undirected)................: 12
PROBEREQUEST (directed)..................: 1
PROBERESPONSE (total)....................: 26
DISASSOCIATION (total)...................: 1
AUTHENTICATION (total)...................: 2
AUTHENTICATION (OPEN SYSTEM).............: 2
ASSOCIATIONREQUEST (total)...............: 1
ASSOCIATIONREQUEST (PSK).................: 1
RESERVED MANAGEMENT frame................: 4
WPA encrypted............................: 280
EAPOL messages (total)...................: 4
EAPOL RSN messages.......................: 4
EAPOLTIME gap (measured maximum msec)....: 4
EAPOL ANONCE error corrections (NC)......: not detected
EAPOL M1 messages (total)................: 1
EAPOL M2 messages (total)................: 1
EAPOL M3 messages (total)................: 1
EAPOL M4 messages (total)................: 1
EAPOL M4 messages (zeroed NONCE).........: 1
EAPOL pairs (total)......................: 3
EAPOL pairs (useful).....................: 3
EAPOL pairs written to 22000 hash file...: 3 (RC checked)
EAPOL M12E2 (challenge)..................: 1
EAPOL M32E2 (authorized).................: 1
EAPOL M32E3 (authorized).................: 1
RSN PMKID (total)........................: 1
RSN PMKID (from zeroed PMK)..............: 1 (not converted by default options - use --all)
RSN PMKID (useful).......................: 1
RSN PMKID written to 22000 hash file.....: 1

frequency statistics from radiotap header (frequency: received packets)
-----------------------------------------------------------------------
 2412: 1093 

Information: limited dump file format detected!
This file format is a very basic format to save captured network data.
It is recommended to use PCAP Next Generation dump file format (or pcapng for short) instead. The PCAP Next Generation dump file format is an attempt to overcome the limitations of the currently widely used (but very limited) libpcap (cap, pcap) format.
https://www.wireshark.org/docs/wsug_html_chunked/AppFiles.html#ChAppFilesCaptureFilesSection
https://github.com/pcapng/pcapng

Information: missing frames!
This dump file does not contain enough EAPOL M1 frames.
It always happens if the capture file was cleaned or it could happen if filter options are used during capturing.
That makes it impossible to calculate nonce-error-correction values.
Duration of the dump tool was a way too short to capture enough additional information.

session summary
---------------
processed cap files...................: 1

OUI information file..........: /home/zerobeat/.hcxtools/oui.txt
OUI entries...................: 35167
total lines read..............: 4
valid hash lines..............: 4
PMKID hash lines..............: 1
EAPOL hash lines..............: 3
filter by status..............: authorized (M1M4, M2M3 or M3M4)
EAPOL written.................: 2

OUI information file..........: /home/zerobeat/.hcxtools/oui.txt
OUI entries...................: 35167
total lines read..............: 4
valid hash lines..............: 4
PMKID hash lines..............: 1
EAPOL hash lines..............: 3
PMKID written.................: 1

$ ls
all.hc22000  authorized.hc22000  pmkid.hc22000  wpa-Induction.pcap
LLH-l commented 2 months ago

If M1 and M3 NONCE are not the same, m23E2 does not necessarily mean authorization, as has been proven in many cases ( Your should also know about it ) Add more detailed hash info is aimed at obtaining better and more valid hashes through bash

e.g

WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00    12.06.2024 15:23:18 261628000   M1M2
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02    12.06.2024 15:23:18 261628000   M2M3
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00    12.06.2024 15:23:18 261628000   M1M2M3
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02    12.06.2024 15:23:18 261628000   M1M2M3

Filter absolute valid hash grep 'M1M2M3' >Filter valid hash cat Filter valid hash

WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00    12.06.2024 15:23:18 261628000   M1M2M3
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02    12.06.2024 15:23:18 261628000   M1M2M3

Obtain human machine readability valid hash ！ awk -F"\t" '{print $1}' Filter valid hash >Valid.hc22000

WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*00
WPA*TYPE*MIC*MACAP*MACCLIENT*ESSID*NONCE*EAPOL*NC*02
ZerBea commented 2 months ago

I guess you're looking for a database tool that handles all your hashes. Such a data base tool is not part of hcxtools. If you need a tool to manage all your hashes (challenge, authorized, PMKID), please take a look at https://wpa-sec.stanev.org/?nets The entire software to set up such a data base is open source and you can find it on git: https://github.com/RealEnder/dwpa

Again the recommended workflow: attack network and make sure the dump file is not crappy if the quality is good, convert to hc22000 file by default options if the dump file is crappy, use --all and get the hash manually feed hashcat with the hc22000 file

And the most important point: Don't waste your time on ancient crappy dump files! Running default options, a crappy dump file results in a crappy hash file -> hashcat fails to recover the PSK Running --all, a crappy dump file results in a crappy hash file too, but the chance to recover the PSK is much better. Manual intervention is mandatory.

If you have a dump file like this one, I recommend to put it in the trash can (where it belongs).

$ hcxpcapngtool crappy.cap
hcxpcapngtool 6.3.4-46-g5377fda reading from crappy.cap...

summary capture file
--------------------
file name................................: crappy.cap
...
ESSID (total unique).....................: 1
BEACON (total)...........................: 1
BEACON on 2.4 GHz channel (from IE_TAG)..: 11 
ACTION (total)...........................: 29
PROBERESPONSE (total)....................: 3207
DEAUTHENTICATION (total).................: 190721
AUTHENTICATION (total)...................: 6
AUTHENTICATION (OPEN SYSTEM).............: 6
REASSOCIATIONREQUEST (total).............: 2
REASSOCIATIONREQUEST (PSK)...............: 2
WPA encrypted............................: 592
EAPOL messages (total)...................: 12
EAPOL RSN messages.......................: 12
EAPOLTIME gap (measured maximum msec)....: 1384
EAPOL ANONCE error corrections (NC)......: not detected
EAPOL M1 messages (total)................: 5
EAPOL M2 messages (total)................: 2
EAPOL M3 messages (total)................: 3
EAPOL M4 messages (total)................: 2
EAPOL M4 messages (zeroed NONCE).........: 2
EAPOL pairs (total)......................: 9
EAPOL pairs (best).......................: 1
EAPOL M32E2 (authorized).................: 1

Information: limited dump file format detected!
This file format is a very basic format to save captured network data.
It is recommended to use PCAP Next Generation dump file format (or pcapng for short) instead. The PCAP Next Generation dump file format is an attempt to overcome the limitations of the currently widely used (but very limited) libpcap (cap, pcap) format.
https://www.wireshark.org/docs/wsug_html_chunked/AppFiles.html#ChAppFilesCaptureFilesSection
https://github.com/pcapng/pcapng

Information: radiotap header is missing!
Radiotap is a de facto standard for 802.11 frame injection and reception. The radiotap header format is a mechanism to supply additional information about frames, from the driver to userspace applications.
https://www.radiotap.org/

Warning: excessive number of deauthentication/disassociation frames detected!
That can cause that an ACCESS POINT change channel, reset EAPOL TIMER, renew ANONCE and set PMKID to zero. This could prevent to calculate a valid EAPOL MESSAGE PAIR, to get a valid PMKID or to decrypt the traffic.

Information: missing frames!
This dump file does not contain undirected proberequest frames.
An undirected proberequest may contain information about the PSK. It always happens if the capture file was cleaned or it could happen if filter options are used during capturing.
That makes it hard to recover the PSK.

Information: no hashes written to hash files

session summary
---------------
processed cap files...................: 1

If you convert it anyway (against all warnings) manual intervention is mandatory.

The second important point: A hash file is not a data base file. If you got the PMK(!) you can remove it. To verify that the PMK is valid, recapture and verify the PMK (hash mode 22001). Everything else is child's play.

The stats of wpa-sec (running hcxtools in background) is impressive: https://wpa-sec.stanev.org/?stats Unfortunately most of the uploads are done by tools (horrible scripts) which doesn't take care about packet losses.

LLH-l commented 2 months ago

 https://github.com/ZerBea/hcxtools/issues/338#issuecomment-2293100482

Regarding 22001 If want to convert a 500GB dictionary to a length of 64 PMK Restoring the PMK key still requires calculating the corresponding PSK  Although I would like to experience your suggestion, it seems that there is a lack of these professional tools, e.g need Convert the 500GB password dictionary to a length of 64 PMK (Or combination format) Calculate the PSK tool corresponding to PMK  I searched a few times and didn't seem to find any about these info  I only know, using m22001, the password dictionary need to be converted to a length of 64 PMK (and the SSID must same to use)

ZerBea commented 2 months ago

Hash mode 22001 is designed to verify an existing PMK or to clean a hash file. Calculating PMKs of an ESSID in combination with a 500GB list does not make sense.

A typical application is:

$ echo "0000000000000000000000000000000000000000000000000000000000000000" > pmk.list
$ cat hashcat.new.pmk | cut -c -64 >> pmk.list
$ hashcat -m 22001 --nonce-error-corrections=32 --remove new_hashfile.hc22000 pmk.list

This will remove known hashes and hashes calculated from a zeroed PMK. I recommend to to this "cleaning" on every new hash file.

Add every new/confirmed PMK to this list.