ZerBea / hcxtools

A small set of tools to convert packets from capture files to hash files for use with Hashcat or John the Ripper.
MIT License
2k stars 392 forks source link

mbedTLS support #106

Closed neheb closed 3 years ago

neheb commented 5 years ago

In OpenWrt, mbedTLS is used for libcurl as the default provider for TLS for size reasons. However with hcxtools, libopenssl must be present as well. This presents a challenge for flash size limited routers. Here are some sizes: https://downloads.openwrt.org/snapshots/packages/arc_arc700/base/

libmbedtls12_2.16.2-1_arc_arc700.ipk | 156.1 KB
libopenssl1.1_1.1.1c-1_arc_arc700.ipk | 855.6 KB

As far as I can tell, the only missing piece for mbedTLS is that CMAC is disabled by default. It can be compile time enabled.

libopenssl is faster, yes, but I don't think speed is important here.

ZerBea commented 5 years ago

Should be not a big problem to use libmbedtls instead of libopenssl. Unfortunately I can't test it here. Can you make a PR?

ZerBea commented 3 years ago

OpenSSL is switching to v3.0.0, soon. Due to some API changes, many functions have to be recoded. Now it's time to think about switching to mbedTLS support.

Is there any good documentation how to replace the following OpenSSL functions by mbedTLS functions:

static inline int mschapv2_challenge_hash(uint8_t peer_challenge, uint8_t auth_challenge, uint8_t username, size_t usernamelen, uint8_t challenge) https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1218

int omac1_aes_128_vector(const uint8_t key, size_t num_elem, const uint8_t addr[], const size_t len, uint8_t mac) https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1667

HMAC(EVP_sha1(), testpmk, 32, pkedata, 100, ptk + p *20, NULL); HMAC(EVP_md5(), &ptk, 16, eapoldata, eapollen, miczero, NULL); HMAC(EVP_sha256(), testpmk, 32, pkedata_prf, 2 + 98 + 2, ptk, NULL); https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1739

PKCS5_PBKDF2_HMAC_SHA1(zeroedpsk, 8, essid, essidlen, 4096, 32, calculatedpmk) https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1790

Any information is appreciated.

neheb commented 3 years ago

curl has some similar code: https://github.com/curl/curl/tree/master/lib

On Mon, Feb 8, 2021 at 2:04 AM ZerBea notifications@github.com wrote:

OpenSSL is switching to v3.0.0, soon. Due to some API changes, many functions have to be recoded. Now it's time to think about switching to mbedTLS support.

Is there any good documentation how to replace the following OpenSSL functions by mbedTLS functions:

static inline int mschapv2_challenge_hash(uint8_t peer_challenge, uint8_t auth_challenge, uint8_t username, size_t usernamelen, uint8_t challenge) https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1218

int omac1_aes_128_vector(const uint8_t key, size_t num_elem, const uint8_t addr[], const size_t len, uint8_t mac) https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1667

HMAC(EVP_sha1(), testpmk, 32, pkedata, 100, ptk + p *20, NULL); HMAC(EVP_md5(), &ptk, 16, eapoldata, eapollen, miczero, NULL); HMAC(EVP_sha256(), testpmk, 32, pkedata_prf, 2 + 98 + 2, ptk, NULL); https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1739

PKCS5_PBKDF2_HMAC_SHA1(zeroedpsk, 8, essid, essidlen, 4096, 32, calculatedpmk) https://github.com/ZerBea/hcxtools/blob/master/hcxpcapngtool.c#L1790

Any information is appreciated.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ZerBea/hcxtools/issues/106#issuecomment-775026671, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWITWI7EBT5OGU4HL7UE5DS56ZK3ANCNFSM4IF33Q5Q .

ZerBea commented 3 years ago

Got them from curl lib directory. Thanks

ZerBea commented 3 years ago

According to this: https://wiki.archlinux.org/index.php/Transport_Layer_Security#Implementations we have several implementations, running Arch Linux. I'll do some tests on some of them (OpenSSL, LibreSSL and mbedTLS) and choose the one which is fast and well documented.

neheb commented 3 years ago

Speed should be in that order.

LibreSSL shouldn't really be used. It's mostly a BSD thing. Alpine Linux removed it because of all the patching that was needed.

mbedTLS is nice for embedded devices. Note that hcxtools are available for OpenWrt where OpenSSL is huge.

ZerBea commented 3 years ago

I aggree. Hcxdumptool and hcxtools are designed to run on small machines (e.g. Raspberry Pi Zero and some small BE systems - you may have noticed the massive code changes during the last past weeks, regarding endianess). That was the reason why I dropped libpcap and WiringPi completely. All the oversized functions are simply not required. Also I'm a friend of the "KISS principle" and I don't like an oversized WiFi suite which can do everything but nothing really prefect. Now I'm going to test the migration of the 6 HMAC functions and the TLS function to mbedTLS. Therefore I have to find a good documention of mbedTLS, without diving into the source code of curl. Unfortunately my first test (pkcs5) using mbedTLS failed. Another option is to take the 6 HMAC functions from hashcat and to use TLS from mbedTLS, only. But anyway, the HMAC functions are mandatory to identify initial PMKIDs/PTKs, calculated from a zeroed PMK. That is the only way to filter them out, because they are useless for us.

BTW: mbedTLS is part of Arch Linux packages: https://archlinux.org/packages/community/x86_64/mbedtls/ https://archlinuxarm.org/packages/arm/mbedtls https://archlinuxarm.org/packages/armv6h/mbedtls https://archlinuxarm.org/packages/armv7h/mbedtls https://archlinuxarm.org/packages/aarch64/mbedtls

Also, I aggree about the installed size: Packages (1) mbedtls-2.25.0-1 Total Installed Size: 5.07 MiB

Packages (1) openssl-1.1.1.i-2 Total Installed Size: 7.31 MiB

neheb commented 3 years ago

On OpenWrt it's 971.2 KB OpenSSL vs 175.1 KB mbedtls for mips architecture.

If you don't need TLS, why use a TLS library? Taking the functions from hashcat sounds like a good idea.

ZerBea commented 3 years ago

We need the TLS functions of the library on EAP-TLS.

Luckily, I think I figured out how the libray works: openssl:

$ wlangenpmk -p passwort -e essid
essid (networkname)....: essid
password...............: passwort
plainmasterkey (SHA1)..: 0897f67a8db79ebadf8b1931d853373ef8b1ec7e80d69f4ecafba220f4d21541

Speedtest 1000 PMKs calculated:
$ time ./wlangenpmk -p passwort -e essid
real    0m1,895s
user    0m1,890s
sys 0m0,000s

vs. mbedTLS:

$ ./mbedtls
0897f67a8db79ebadf8b1931d853373ef8b1ec7e80d69f4ecafba220f4d21541

Speedtest 1000 PMKs calculated:
$ time ./mbedtls
real    0m7,826s
user    0m7,822s
sys 0m0,000s

After some speed tests, I noticed that the single thread calculation time is much more than expected. We are nearly 4 times slower than running openssl. That will slow down the conversion process of pcapngfiles (containig many EAPOL messages and/or PMKIDs) extremely.

neheb commented 3 years ago

Unfortunate.

There's also WolfSSL, which is smaller than OpenSSL and around 2x mbedTLS. It should be comparable to OpenSSL.

ZerBea commented 3 years ago

Indeed, unfortunate. Everything looked fine, until a started the speed comparison.

WolfSSL is no alternative, because it isn't part of Arch Linux package system.

Looks like there will be no quick and easy solution. But anyway, I started to add openssl 3.0 compatibility to hcxtools to avoid tons of warnings when Arch moved to 3.0, until I'll find a better solution (adding hashcat HMAC functions on time sensitive algo's).

ZerBea commented 3 years ago

Closed this feature request due to massive speed impact. Now only 2 options remaining: adding openssl 3.0.0 support - quick fix, only few test mandatory adding hashcat has functions - will take a while, massive tests mandatory

ZerBea commented 3 years ago

Looks like this patch is working and we have a tiny alternative, now: https://github.com/ZerBea/hcxtools/issues/184#issuecomment-799430178 reference: https://github.com/libressl-portable/portable/issues/651