search

openwall / john

John the Ripper jumbo - advanced offline password cracker, which supports hundreds of hash and cipher types, and runs on many operating systems, CPUs, GPUs, and even some FPGAs
https://www.openwall.com/john/
Other
10.29k stars 2.1k forks source link

Many tests fail with Mesa #4108

Open GKNSB opened 5 years ago

GKNSB commented 5 years ago

Hello everyone. First of, here are some information 

# ./john --list=build-info
Version: 1.9.0-jumbo-1+bleeding-ea33667eb 2019-09-23 12:43:34 +0200
Build: linux-gnu 64-bit x86_64 SSE2 AC OMP
SIMD: SSE2, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
$JOHN is ./
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 2147483648
SINGLE_BUF_MAX: 4294967295
Effective limit: Number of salts vs. SingleMaxBufferSize
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 8.3.0
GNU libc version: 2.28 (loaded: 2.28)
OpenCL headers version: 2.0
Crypto library: OpenSSL
OpenSSL library version: 01010104f
OpenSSL 1.1.1d  10 Sep 2019
GMP library version: 6.1.2
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
times(2) sysconf(_SC_CLK_TCK) is 100
Using times(2) for timers, resolution 10 ms
HR timer claimed resolution 1 ns, observed 1 us
Total physical host memory: 3947 MB
Available physical host memory: 3353 MB
# ./john --list=opencl-devices
Platform #0 name: Clover, version: OpenCL 1.1 Mesa 18.3.6
    Device #0 (1) name:     AMD PITCAIRN (DRM 2.50.0, 4.19.0-6-amd64, LLVM 7.0.1)
    Device vendor:          AMD
    Device type:            GPU (LE)
    Device version:         OpenCL 1.1 Mesa 18.3.6
    Driver version:         18.3.6
    Native vector widths:   char 16, short 8, int 4, long 2
    Preferred vector width: char 16, short 8, int 4, long 2
    Global Memory:          2043 MB
    Local Memory:           32 KB (Local)
    Constant Buffer size:   716 MB
    Max memory alloc. size: 716 MB
    Max clock (MHz):        900
    Max Work Group Size:    256
    Parallel compute cores: 16
    Stream processors:      1024  (16 x 64)
    Speed index:            921600

After having installed what in my head looks like the requirements to provide opencl support for my Saphire HD 7850, john failes on opencl tests. More specifically, rar-opencl seems to break the test loop altogether.

# ./john --test --format=opencl
Device 1: AMD PITCAIRN (DRM 2.50.0, 4.19.0-6-amd64, LLVM 7.0.1)
Benchmarking: sha1crypt-opencl, (NetBSD) [PBKDF1-SHA1 OpenCL 4x]... Error: 4 unsupported relocations
Error: 4 unsupported relocations
Error: 4 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_sha1crypt_fmt_plug.c:277 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: KeePass-opencl [SHA256 AES/Twofish/ChaCha OpenCL]... Error: 74 unsupported relocations
Error: 74 unsupported relocations
Error: 74 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_keepass_fmt_plug.c:263 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: oldoffice-opencl, MS Office <= 2003 [MD5/SHA1 RC4 OpenCL/mask accel]... Error: 4 unsupported relocations
FAILED (cmp_one(0))
Benchmarking: PBKDF2-HMAC-MD4-opencl [PBKDF2-MD4 OpenCL 4x]... Error: 4 unsupported relocations
Error: 4 unsupported relocations
Error: 4 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_pbkdf2_hmac_md4_fmt_plug.c:279 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: PBKDF2-HMAC-MD5-opencl [PBKDF2-MD5 OpenCL 4x]... Error: 4 unsupported relocations
Error: 4 unsupported relocations
Error: 4 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_pbkdf2_hmac_md5_fmt_plug.c:276 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: PBKDF2-HMAC-SHA1-opencl [PBKDF2-SHA1 OpenCL 4x]... Error: 2 unsupported relocations
Error: 2 unsupported relocations
Error: 2 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_pbkdf2_hmac_sha1_fmt_plug.c:300 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: rar-opencl, RAR3 (length 5) [SHA1 OpenCL AES]... (2xOMP) Error: 6 unsupported relocations
Error: 6 unsupported relocations
Error: 6 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_rar_fmt_plug.c:334 - Error running loop kernel
FAILED (cmp_all(-1))
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_rar_fmt_plug.c:220 - Error releasing memory mappings

Here is another one for testing nt-opencl:

# ./john --test --format=nt-opencl
Device 1: AMD PITCAIRN (DRM 2.50.0, 4.19.0-6-amd64, LLVM 7.0.1)
Benchmarking: NT-opencl [MD4 OpenCL/mask accel]... Build log: input.cl:332:18: warning: unknown attribute 'max_constant_size' ignored

Build log: input.cl:332:18: warning: unknown attribute 'max_constant_size' ignored

0: OpenCL CL_INVALID_EVENT (-58) error in opencl_common.c:1801 - clWaitForEvents

And when I tried against a small test file of ntlm hashes:

# ./john ~/test-ntlm.txt --format=nt-opencl
Device 1: AMD PITCAIRN (DRM 2.50.0, 4.19.0-6-amd64, LLVM 7.0.1)
Using default input encoding: UTF-8
Loaded 32 password hashes with no different salts (NT-opencl [MD4 OpenCL])
Build log: input.cl:332:18: warning: unknown attribute 'max_constant_size' ignored

Self test failed (cmp_one(0))

Version of john is latest jumbo cloned from github. I cannot figure out if it is an issue with my installation, or from the related issues I've seen with previous versions. I remain at your disposal should you require any further information regarding the issue.

magnumripper commented 5 years ago

We do mention Mesa in doc/README-OPENCL. While it may eventually become stable enough that we could start working around remaining problems, they are not there yet. Your best bet is trying a proprietary driver but AMD drivers have their share of problems as well, unfortunately...

magnumripper commented 5 years ago

Then again you could of course report a bug with them instead!

GKNSB commented 5 years ago

True, I just wanted to let you guys know about the latest status. I'll look into it more and update the issue accordingly even with proprietary drivers so that there is a reference for anyone else interested. Thanks a lot for the reply @magnumripper. Cheers

solardiz commented 2 years ago

According to @AlekseyCherepanov, almost half of our OpenCL formats now work with Mesa:

https://www.openwall.com/lists/john-users/2021/11/27/1

See also https://github.com/openwall/john/issues/3610#issuecomment-982652301 for the much better status with amdgpu-pro.

AlekseyCherepanov commented 2 years ago

@GKNSB DRM 2.50.0 in the output suggests that you use radeon driver. You might try to switch to amdgpu driver (it should be already installed).

I used this advice to switch driver. (.si / .cik seem messed up there. Gentoo's wiki suggests pitcairn belongs to Southern Islands family. [Arch's wiki] suggests Southern Islands use .si_support argument.)

Also I tried blacklisting radeon driver as described here. It did not work for me.

You may check which driver is active using lspci -v:

[...]
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii PRO [Radeon R9 290/390] (rev 80) (prog-if 00 [VGA controller])
[...]
        Kernel driver in use: amdgpu
        Kernel modules: radeon, amdgpu
[...]

If none of formats work at all, you may need to install non-free package with firmware for gpu. I installed it as dependency of broad package firmware-linux on Debian.

AlekseyCherepanov commented 2 years ago

On mesa krb5asrep-aes-opencl and sappse-opencl formats get stuck during --test.

^C or kill -9 cannot quit john. Reboot with command reboot helps but takes unusual 5+ minutes. Before ^C it is possible to use clinfo. After ^C clinfo gets stuck too.

(Previously with amdgpu-pro I had gpu stuck and reboot was not able to reboot machine actually, so I had to use reset button. I had older kernel, so I guess the ability to reboot is not question of mesa vs amdgpu-pro.)

krb5asrep-aes-opencl gets stuck during auto-tuning. Setting LWS/GWS manually bypasses this problem.

$ ./run/john --test --format=krb5asrep-aes-opencl --verbosity=6
initUnicode(UNICODE, RAW/RAW)
RAW -> RAW -> RAW
Device 1: AMD Radeon (TM) R9 390 Series (HAWAII, DRM 3.40.0, 5.10.0-9-amd64, LLVM 11.0.1)
Benchmarking: krb5asrep-aes-opencl, Kerberos 5 AS-REP etype 17/18 [PBKDF2-SHA1 OpenCL 4x]... Loaded 3 hashes with 3 different salts to test db from test vectors
TestDB LWS=7 GWS=49 (7 blocks) PASS, 
Test mask: ?a?a?l?u?d?d?s
Calculating best GWS for LWS=64; max. 100 ms single kernel invocation.
Raw speed figures including buffer transfers:
Tuning for password length 7
xfer: 445 ns, init: 356.593 us, loop: 78x6.937 ms, final: 241.037 us, asrep: 1.420 ms, res xfer: 161.333 us
gws:       128      942 c/s    15433728 rounds/s  543.447 ms per crypt_all()!
xfer: 445 ns, init: 357.926 us, loop: 78x7.277 ms, final: 241.926 us, asrep: 1.783 ms, res xfer: 135.259 us
gws:       256     1795 c/s    29409280 rounds/s  570.289 ms per crypt_all()+
xfer: 445 ns, init: 442.963 us, loop: 78x7.308 ms, final: 313.926 us, asrep: 2.428 ms, res xfer: 145.334 us
gws:       512     3570 c/s    58490880 rounds/s  573.564 ms per crypt_all()+
xfer: 296 ns, init: 888 us, loop: 78x7.263 ms, final: 666.963 us, asrep: 3.718 ms, res xfer: 159.111 us
gws:      1024     7158 c/s   117276672 rounds/s  572.151 ms per crypt_all()!
xfer: 297 ns, init: 1.292 ms, loop: 78x7.293 ms, final: 851.703 us, asrep: 6.710 ms, res xfer: 179.703 us
gws:      2048    14170 c/s   232161280 rounds/s  578.087 ms per crypt_all()+
xfer: 445 ns, init: 2.376 ms, loop: 78x7.597 ms, final: 1.515 ms, asrep: 20.263 ms, res xfer: 218.074 us
gws:      4096    26550 c/s   434995200 rounds/s  617.087 ms per crypt_all()+
xfer: 444 ns, init: 5.109 ms, loop: 78x7.782 ms, final: 1.553 ms, asrep: 6.140 ms, res xfer: 257.186 us
gws:      8192    52829 c/s   865550336 rounds/s  620.257 ms per crypt_all()+
xfer: 296 ns, init: 1.113 ms, loop: 78x14.515 ms, final: 852.148 us, asrep: 10.199 ms, res xfer: 397.630 us
gws:     16384    57233 c/s   937705472 rounds/s     1.145 s per crypt_all()+
xfer: 445 ns, init: 1.946 ms, loop: 78x28.774 ms, final: 1.520 ms, asrep: 17.614 ms, res xfer: 457.777 us
gws:     32768    57829 c/s   947470336 rounds/s     2.266 s per crypt_all()+
xfer: 296 ns, init: 3.926 ms, loop: 78x50.503 ms, final: 3.013 ms, asrep: 31.419 ms, res xfer: 693.926 us
gws:     65536    65876 c/s  1079312384 rounds/s     3.979 s per crypt_all()+
xfer: 296 ns, init: 7.755 ms, loop: 78x93.978 ms, final: 5.982 ms, asrep: 59.280 ms, res xfer: 1.260 ms
gws:    131072    70788 c/s  1159790592 rounds/s     7.406 s per crypt_all()+

sappse-opencl gets stuck before auto-tuning. (Setting LWS/GWS did not help. I tried values 1/1 and 16/4096.)

$ time ./run/john --test --format=sappse-opencl --verbosity=6
initUnicode(UNICODE, RAW/RAW)
RAW -> RAW -> RAW
Device 1: AMD Radeon (TM) R9 390 Series (HAWAII, DRM 3.40.0, 5.10.0-9-amd64, LLVM 11.0.1)
Benchmarking: sappse-opencl, SAP PSE [PKCS#12 PBE (SHA1) OpenCL]... Loaded 5 hashes with 5 different salts to test db from test vectors
TestDB LWS=7 GWS=49 (7 blocks)
AlekseyCherepanov commented 2 years ago

Somehow nt-opencl fail self-tests starting real attack, while --test reports success.

$ ./john/run/john --format=nt-opencl --device=1 --test=0
Device 1: AMD Radeon (TM) R9 390 Series (HAWAII, DRM 3.40.0, 5.10.0-9-amd64, LLVM 11.0.1)
Testing: NT-opencl [MD4 OpenCL]... PASS
$ ./john/run/john --format=nt-opencl --device=1 --test
Device 1: AMD Radeon (TM) R9 390 Series (HAWAII, DRM 3.40.0, 5.10.0-9-amd64, LLVM 11.0.1)
Benchmarking: NT-opencl [MD4 OpenCL/mask accel]... Build log: input.cl:332:18: warning: unknown attribute 'max_constant_size' ignored [-Wunknown-attributes]

LWS=256 GWS=163840 (640 blocks) x8580 DONE
Raw:    9742M c/s real, 196804M c/s virtual
$ ./john/run/john --format=nt-opencl --device=1 --mask='?a?a?a?a?a?a?a' <(echo aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)
Device 1: AMD Radeon (TM) R9 390 Series (HAWAII, DRM 3.40.0, 5.10.0-9-amd64, LLVM 11.0.1)
Using default input encoding: UTF-8
Loaded 1 password hash (NT-opencl [MD4 OpenCL])
Self test failed (cmp_one(0))

It seems wrong that self test failed before real attack but it did not fail with --test.

AlekseyCherepanov commented 2 years ago

JFYI if mesa is present, john built with asan fails to start (local paths are replaced by ...):

$ ./run/john --test --format=odf-opencl --device=2
NOTE: This is a debug build, speed will be lower than normal
==124564==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/asan/asan_interceptors.cpp:333 "((__interception::real___cxa_throw)) != (0)" (0x0, 0x0)
    #0 0x7f41d8158657 in AsanCheckFailed ../../../../src/libsanitizer/asan/asan_rtl.cpp:73
    #1 0x7f41d8175d4a in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) ../../../../src/libsanitizer/sanitizer_common/sanitizer_termination.cpp:78
    #2 0x7f41d80dec14 in __interceptor___cxa_throw ../../../../src/libsanitizer/asan/asan_interceptors.cpp:333
    #3 0x7f41d475627e  (/lib/x86_64-linux-gnu/libMesaOpenCL.so.1+0x2527e)
    #4 0x7f41d4783db7  (/lib/x86_64-linux-gnu/libMesaOpenCL.so.1+0x52db7)
    #5 0x7f41d4759904  (/lib/x86_64-linux-gnu/libMesaOpenCL.so.1+0x28904)
    #6 0x7f41d8aa2fe1 in call_init elf/dl-init.c:72
    #7 0x7f41d8aa30e8 in call_init elf/dl-init.c:30
    #8 0x7f41d8aa30e8 in _dl_init elf/dl-init.c:119
    #9 0x7f41d7af42bc in __GI__dl_catch_exception elf/dl-error-skeleton.c:182
    #10 0x7f41d8aa7057 in dl_open_worker elf/dl-open.c:758
    #11 0x7f41d7af425f in __GI__dl_catch_exception elf/dl-error-skeleton.c:208
    #12 0x7f41d8aa68f9 in _dl_open elf/dl-open.c:837
    #13 0x7f41d7bb7257 in dlopen_doit dlfcn/dlopen.c:66
    #14 0x7f41d7af425f in __GI__dl_catch_exception elf/dl-error-skeleton.c:208
    #15 0x7f41d7af431e in __GI__dl_catch_error elf/dl-error-skeleton.c:227
    #16 0x7f41d7bb7a64 in _dlerror_run dlfcn/dlerror.c:170
    #17 0x7f41d7bb72e3 in __dlopen dlfcn/dlopen.c:87
    #18 0x7f41d80f06d5 in __interceptor_dlopen ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:6042
    #19 0x7f41d7d25678  (/lib/x86_64-linux-gnu/libOpenCL.so.1+0x6678)
    #20 0x7f41d7d25875  (/lib/x86_64-linux-gnu/libOpenCL.so.1+0x6875)
    #21 0x7f41d7d2674d in clGetPlatformIDs (/lib/x86_64-linux-gnu/libOpenCL.so.1+0x774d)
    #22 0x557af019a83d in load_opencl_environment .../src/opencl_common.c:432
    #23 0x557af01a36f5 in opencl_load_environment .../src/opencl_common.c:850
    #24 0x557af00bc6ce in benchmark_all .../src/bench.c:766
    #25 0x557af00ef71c in john_run .../src/john.c:1661
    #26 0x557af00ef71c in main .../src/john.c:2082
    #27 0x7f41d79e2d09 in __libc_start_main ../csu/libc-start.c:308
    #28 0x557aefbf1489 in _start (.../run/john+0x2da489)
AlekseyCherepanov commented 2 years ago

It seems wrong that self test failed before real attack but it did not fail with --test.

Real attack implies --encoding=utf-8. It may be specified explicitly for --test. It might explain the difference. But I cannot try it now.