search

openwall / john

John the Ripper jumbo - advanced offline password cracker, which supports hundreds of hash and cipher types, and runs on many operating systems, CPUs, GPUs, and even some FPGAs
https://www.openwall.com/john/
Other
10.02k stars 2.07k forks source link

POCL testing (Ubuntu 20.04) #4684

Closed magnumripper closed 3 years ago

magnumripper commented 3 years ago

Our current README-OPENCL says POCL is not supported (but will work with some formats). I'll do some testing and see current state, and perhaps come up with some tweaks. Perhaps we can stop saying it's not supported?

This is POCL 1.4

Platform #0 name: Portable Computing Language, version: OpenCL 1.2 pocl 1.4, None+Asserts, LLVM 9.0.1, RELOC, SLEEF, DISTRO, POCL_DEBUG
    Device #0 (1) name:     pthread-Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
    Device vendor:          GenuineIntel
    Device type:            CPU (LE)
    Device version:         OpenCL 1.2 pocl HSTR: pthread-x86_64-pc-linux-gnu-haswell
    Driver version:         1.4 
    Native vector widths:   char 16, short 16, int 8, long 4
    Preferred vector width: char 16, short 16, int 8, long 4
    Global Memory:          30005 MiB
    Global Memory Cache:    8 MiB
    Local Memory:           4 MiB (Global)
    Constant Buffer size:   4 MiB
    Max memory alloc. size: 8 GiB
    Max clock (MHz):        4000
    Profiling timer res.:   1 ns
    Max Work Group Size:    4096
    Parallel compute cores: 8
    Speed index:            128000

The ones that fail or warn:

Device 1: pthread-Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
Testing: sha256crypt-opencl, crypt(3) $5$ (rounds=5000) [SHA256 OpenCL]... 1 warning generated.
Build log: warning: /home/magnum/.cache/pocl/kcache/tempfile-14-89-0e-36-c9.cl:265:13: loop not unrolled: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering

PASS
Testing: sha512crypt-opencl, crypt(3) $6$ (rounds=5000) [SHA512 OpenCL]... 1 warning generated.
Build log: warning: /home/magnum/.cache/pocl/kcache/tempfile-0c-de-2e-e2-26.cl:232:13: loop not unrolled: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering

PASS
Testing: descrypt-opencl, traditional crypt(3) [DES OpenCL]... 0: OpenCL CL_INVALID_GLOBAL_WORK_SIZE (-63) error in opencl_DES_bs_b_plug.c:683 - Enque kernel DES_bs_25 failed.

Testing: gpg-opencl, OpenPGP / GnuPG Secret Key [SHA1/SHA2 OpenCL]... (8xOMP) Segmentation fault (core dumped)

All others pass!

magnumripper commented 3 years ago

I see it vectorizes 8x for AVX2, that's great (considering some other CPU drivers settles for 4x even though AVX2 is available). But that's just my vector-aware formats - I have no idea if it can auto-vectorize.

solardiz commented 3 years ago

Perhaps we can stop saying it's not supported?

Looks so, from your testing. Perhaps update the documentation? Ideally, also try and get descrypt-opencl to work. Thank you!

magnumripper commented 3 years ago

This part of auto-tune for ODF-opencl isn't great, some tweak needed.

Calculating best LWS for GWS=8192
Testing LWS=8 GWS=8192 ... 418.048 ms+
Testing LWS=16 GWS=8192 ... 11.513 s
Testing LWS=32 GWS=8192 ... 11.510 s
Testing LWS=64 GWS=8192 ... 11.553 s
Testing LWS=128 GWS=8192 ... 11.547 s
Testing LWS=256 GWS=8192 ... 11.609 s
Testing LWS=512 GWS=8192 ... 11.631 s
Testing LWS=1024 GWS=8192 ... 11.752 s
Testing LWS=2048 GWS=8192 ... 11.930 s
Testing LWS=4096 GWS=8192 ... 12.004 s
magnumripper commented 3 years ago

Good thing I found that --force-scalar was inadvertently disallowed with --test. After that,

Vector:

Device 1: pthread-Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
Benchmarking: sha1crypt-opencl, (NetBSD) [PBKDF1-SHA1 OpenCL 8x]... LWS=8 GWS=128 (16 blocks) DONE
Speed for cost 1 (iteration count) of 20000
Raw:    2216 c/s real, 863 c/s virtual

Scalar:

Device 1: pthread-Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
Benchmarking: sha1crypt-opencl, (NetBSD) [PBKDF1-SHA1 OpenCL]... LWS=8 GWS=128 (16 blocks) DONE
Speed for cost 1 (iteration count) of 20000
Raw:    357 c/s real, 150 c/s virtual

CPU format:

Benchmarking: sha1crypt, NetBSD's sha1crypt [PBKDF1-SHA1 256/256 AVX2 8x]... (8xOMP) DONE
Speed for cost 1 (iteration count) of 20000
Raw:    2461 c/s real, 812 c/s virtual

If this is due to OMP and large KPC, I should be able to get there with higher GWS... yep:

$ ./john -test -form:sha1crypt-opencl -dev=cpu -lws=8 -gws=512
Device 1: pthread-Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
Benchmarking: sha1crypt-opencl, (NetBSD) [PBKDF1-SHA1 OpenCL 8x]... LWS=8 GWS=512 (64 blocks) DONE
Speed for cost 1 (iteration count) of 20000
Raw:    2512 c/s real, 877 c/s virtual
magnumripper commented 3 years ago

I didn't look at speeds at all apart from the above, but from what I've seen today POCL is mature and good stuff, especially knowing how much problems there were a couple of years ago.

magnumripper commented 3 years ago

POCL is mature and good stuff

Benhmarking formats lacking any vector magic, it looks like POCL doesn't have any auto-vectorization at all though. Anyone stuck with CPU should find a good Intel driver that does that.

frank-dittrich commented 3 years ago

I decided to add my Fedora (5.11.15-100.fc32.x86_64) results here instead of creating a new, fedora specific issue.

$ ./john --test=0 --device=2 --format=descrypt-opencl
Device 2: pthread-Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
Testing: descrypt-opencl, traditional crypt(3) [DES OpenCL]... 0: OpenCL CL_INVALID_GLOBAL_WORK_SIZE (-63) error in opencl_DES_bs_b_plug.c:683 - Enque kernel DES_bs_25 failed.

However:

$ GWS=128 ./john --test --device=2 --format=descrypt-opencl
Device 2: pthread-Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
Benchmarking: descrypt-opencl, traditional crypt(3) [DES OpenCL/mask accel]... LWS=32 GWS=128 DONE
Many salts: 1476K c/s real, 212110 c/s virtual
Only one salt:  1503K c/s real, 196954 c/s virtual

Afterwards, the previously failing test passes as well:

$ ./john --test=0 --device=2 --format=descrypt-opencl
Device 2: pthread-Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
Testing: descrypt-opencl, traditional crypt(3) [DES OpenCL]... PASS

And a segfault in gpg-opencl:

$ gdb ./john
GNU gdb (GDB) Fedora 9.1-7.fc32
[...]
(gdb) run --test --device=2 --format=gpg-opencl
Starting program: /home/fd/git/john/run/john --test --device=2 --format=gpg-opencl
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Will run 8 OpenMP threads
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
[New Thread 0x7fffdc701700 (LWP 13734)]
[New Thread 0x7fffdbf00700 (LWP 13735)]
[New Thread 0x7fffdb6ff700 (LWP 13736)]
[New Thread 0x7fffdaefe700 (LWP 13737)]
[New Thread 0x7fffda6fd700 (LWP 13738)]
[New Thread 0x7fffd9efc700 (LWP 13739)]
[New Thread 0x7fffd96fb700 (LWP 13740)]
[New Thread 0x7fffd8efa700 (LWP 13741)]
[New Thread 0x7fffbbfff700 (LWP 13742)]
[New Thread 0x7fffbb7fe700 (LWP 13743)]
[New Thread 0x7fffbaffd700 (LWP 13744)]
[New Thread 0x7fffba7fc700 (LWP 13745)]
[New Thread 0x7fffb9ffb700 (LWP 13746)]
[New Thread 0x7fffb97fa700 (LWP 13747)]
[New Thread 0x7fffb8ff9700 (LWP 13748)]
[New Thread 0x7fff97fff700 (LWP 13749)]
[New Thread 0x7fff96f2a700 (LWP 13750)]
[New Thread 0x7fff96729700 (LWP 13751)]
[New Thread 0x7fff95f28700 (LWP 13752)]
[New Thread 0x7fff95727700 (LWP 13753)]
[Thread 0x7fff97fff700 (LWP 13749) exited]
[Thread 0x7fffb8ff9700 (LWP 13748) exited]
[Thread 0x7fffb97fa700 (LWP 13747) exited]
[Thread 0x7fffb9ffb700 (LWP 13746) exited]
[Thread 0x7fffba7fc700 (LWP 13745) exited]
[Thread 0x7fffbaffd700 (LWP 13744) exited]
[Thread 0x7fffbb7fe700 (LWP 13743) exited]
[Thread 0x7fffbbfff700 (LWP 13742) exited]
[Thread 0x7fffd8efa700 (LWP 13741) exited]
[Thread 0x7fffd96fb700 (LWP 13740) exited]
[Thread 0x7fffd9efc700 (LWP 13739) exited]
[Thread 0x7fffda6fd700 (LWP 13738) exited]
[Thread 0x7fffdaefe700 (LWP 13737) exited]
[Thread 0x7fff96729700 (LWP 13751) exited]
[Thread 0x7fff95727700 (LWP 13753) exited]
[Thread 0x7fff95f28700 (LWP 13752) exited]
[Thread 0x7fff96f2a700 (LWP 13750) exited]
[Thread 0x7fffdb6ff700 (LWP 13736) exited]
[Thread 0x7fffdbf00700 (LWP 13735) exited]
[Thread 0x7fffdc701700 (LWP 13734) exited]
[New Thread 0x7fff95727700 (LWP 13754)]
[Thread 0x7fff95727700 (LWP 13754) exited]
[New Thread 0x7fff95727700 (LWP 13757)]
[New Thread 0x7fff95f28700 (LWP 13758)]
[New Thread 0x7fff96729700 (LWP 13759)]
[New Thread 0x7fff96f2a700 (LWP 13760)]
[New Thread 0x7fffd9a11700 (LWP 13761)]
[New Thread 0x7fffd9210700 (LWP 13762)]
[New Thread 0x7fffb8ffc700 (LWP 13763)]
[New Thread 0x7fff97fff700 (LWP 13764)]
Device 2: pthread-Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
Benchmarking: gpg-opencl, OpenPGP / GnuPG Secret Key [SHA1/SHA2 OpenCL]... (8xOMP) -
Thread 26 "john" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff96f2a700 (LWP 13760)]
0x00007fffea6a6df3 in S2KItSaltedSHA1Generator () from /home/fd/.cache/pocl/kcache/IM/IFIFGODMBODIGGJIHBIKDGIPGIKNAODEJHBKN/gpg/7-1-1-goffs0-smallgrid/gpg.so
Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.8-2.fc32.x86_64 clang-libs-10.0.1-3.fc32.x86_64 elfutils-libelf-0.183-1.fc32.x86_64 expat-2.2.8-2.fc32.x86_64 gmp-6.1.2-13.fc32.x86_64 hwloc-libs-2.0.4-3.fc32.x86_64 libX11-1.6.12-1.fc32.x86_64 libXau-1.0.9-3.fc32.x86_64 libXext-1.3.4-3.fc32.x86_64 libXfixes-5.0.3-11.fc32.x86_64 libdrm-2.4.102-1.fc32.x86_64 libedit-3.1-32.20191231cvs.fc32.x86_64 libffi-3.1-24.fc32.x86_64 libglvnd-1.3.2-1.--Type <RET> for more, q to quit, c to continue without paging--
fc32.x86_64 libglvnd-glx-1.3.2-1.fc32.x86_64 libpciaccess-0.16-2.fc32.x86_64 libtool-ltdl-2.4.6-33.fc32.x86_64 libxcb-1.13.1-4.fc32.x86_64 libxcrypt-4.4.19-1.fc32.x86_64 llvm-libs-10.0.1-4.fc32.x86_64 mesa-dri-drivers-20.2.3-1.fc32.x86_64 mesa-libOpenCL-20.2.3-1.fc32.x86_64 ocl-icd-2.2.13-1.fc32.x86_64 openssl-libs-1.1.1k-1.fc32.x86_64 sssd-client-2.4.0-1.fc32.x86_64 zlib-1.2.11-21.fc32.x86_64
(gdb) bt
#0  0x00007fffea6a6df3 in S2KItSaltedSHA1Generator () from /home/fd/.cache/pocl/kcache/IM/IFIFGODMBODIGGJIHBIKDGIPGIKNAODEJHBKN/gpg/7-1-1-goffs0-smallgrid/gpg.so
#1  0x3433323130393837 in ?? ()
#2  0x3231303938373635 in ?? ()
#3  0x3039383736353433 in ?? ()
#4  0x00007f3534333231 in ?? ()
#5  0x00007fff80001680 in ?? ()
#6  0x00007fff96f29d30 in ?? ()
#7  0x0000000000000001 in ?? ()
#8  0x00007fff96f29e70 in ?? ()
#9  0x00007fffdcadc377 in pocl_pthread_driver_thread () from /lib64/libpocl.so.2.5.0
Backtrace stopped: Cannot access memory at address 0x3635343332313041

All other formats pass self test on device 2 (CPU).

solardiz commented 3 years ago

@frank-dittrich Can you retest now that #4686 is merged? Thank you!

frank-dittrich commented 3 years ago

For

Device 2: pthread-Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz

the 2 formats that caused problems now work. Will do a full test as well, but first I want to report the problems with the "GPU".

Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Benchmarking: dmg-opencl, Apple DMG [PBKDF2-SHA1 3DES/AES OpenCL 4x]...   %1072 = bitcast i24 %1071 to <3 x i8>
unhandled instruction
UNREACHABLE executed at /builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp:619!
Aborted (core dumped)
Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Benchmarking: EncFS-opencl [PBKDF2-SHA1 AES OpenCL 4x]...   %1072 = bitcast i24 %1071 to <3 x i8>
unhandled instruction
UNREACHABLE executed at /builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp:619!
Aborted (core dumped)
Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Benchmarking: krb5pa-sha1-opencl, Kerberos 5 AS-REQ Pre-Auth etype 17/18 [PBKDF2-SHA1 OpenCL 4x]...   %1075 = bitcast i24 %1074 to <3 x i8>
unhandled instruction
UNREACHABLE executed at /builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp:619!
Aborted (core dumped)
Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Benchmarking: krb5asrep-aes-opencl, Kerberos 5 AS-REP etype 17/18 [PBKDF2-SHA1 OpenCL 4x]...   %1075 = bitcast i24 %1074 to <3 x i8>
unhandled instruction
UNREACHABLE executed at /builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp:619!
Aborted (core dumped)
Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Benchmarking: OpenBSD-SoftRAID-opencl [PBKDF2-SHA1 AES OpenCL 4x]...   %1072 = bitcast i24 %1071 to <3 x i8>
unhandled instruction
UNREACHABLE executed at /builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp:619!
Aborted (core dumped)
Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Benchmarking: pfx-opencl, (.pfx, .p12) [PKCS#12 PBE (SHA1/SHA-256/512) OpenCL]...   %1075 = bitcast i24 %1074 to <3 x i8>
unhandled instruction
UNREACHABLE executed at /builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp:619!
Aborted (core dumped)
Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Benchmarking: ZIP-opencl, WinZip [PBKDF2-SHA1 OpenCL]...   %1075 = bitcast i24 %1074 to <3 x i8>
unhandled instruction
UNREACHABLE executed at /builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp:619!
Aborted (core dumped)

These all look similar, that's why just one stack trace:

Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Testing: ZIP-opencl, WinZip [PBKDF2-SHA1 OpenCL]...   %1075 = bitcast i24 %1074 to <3 x i8>
unhandled instruction
UNREACHABLE executed at /builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp:619!

Thread 1 "john" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50    return ret;
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff779b895 in __GI_abort () at abort.c:79
#2  0x00007ffff20eab6e in llvm::llvm_unreachable_internal (msg=msg@entry=0x7ffff70ed62f "unhandled instruction", file=file@entry=0x7ffff70ed188 "/builddir/build/BUILD/Beignet-1.3.2-Source/backend/src/llvm/PromoteIntegers.cpp", 
    line=line@entry=619) at ../lib/Support/ErrorHandling.cpp:189
#3  0x00007ffff6fca298 in convertInstruction (State=..., Inst=0x42f9708) at /usr/src/debug/beignet-1.3.2-5.fc29.x86_64/backend/src/llvm/PromoteIntegers.cpp:619
#4  (anonymous namespace)::PromoteIntegers::runOnFunction (this=<optimized out>, F=...) at /usr/src/debug/beignet-1.3.2-5.fc29.x86_64/backend/src/llvm/PromoteIntegers.cpp:647
#5  0x00007ffff22359b8 in llvm::FPPassManager::runOnFunction (F=..., this=0x426f5f0) at ../lib/IR/LegacyPassManager.cpp:1524
#6  llvm::FPPassManager::runOnFunction (this=this@entry=0x426f5f0, F=...) at ../lib/IR/LegacyPassManager.cpp:1498
#7  0x00007ffff2d74761 in (anonymous namespace)::CGPassManager::RunPassOnSCC (DevirtualizedCall=<synthetic pointer>: <optimized out>, CallGraphUpToDate=<synthetic pointer>: <optimized out>, CG=..., CurSCC=..., P=0x426f5f0, 
    this=0x452fb60) at ../lib/Analysis/CallGraphSCCPass.cpp:156
#8  (anonymous namespace)::CGPassManager::RunAllPassesOnSCC (DevirtualizedCall=<synthetic pointer>: <optimized out>, CG=..., CurSCC=..., this=0x452fb60) at ../lib/Analysis/CallGraphSCCPass.cpp:423
#9  (anonymous namespace)::CGPassManager::runOnModule (this=0x452fb60, M=...) at ../lib/Analysis/CallGraphSCCPass.cpp:479
#10 0x00007ffff2235212 in (anonymous namespace)::MPPassManager::runOnModule (M=..., this=0x4299c10) at ../lib/IR/LegacyPassManager.cpp:1597
#11 llvm::legacy::PassManagerImpl::run (this=0x47faf50, M=...) at ../lib/IR/LegacyPassManager.cpp:1700
#12 0x00007ffff6fdf88b in gbe::llvmToGen (unit=..., module=module@entry=0x3f861d0, optLevel=optLevel@entry=1, strictMath=strictMath@entry=true, profiling=<optimized out>, errors="")
    at /usr/src/debug/beignet-1.3.2-5.fc29.x86_64/backend/src/llvm/llvm_to_gen.cpp:399
#13 0x00007ffff6f49dc1 in gbe::Program::buildFromLLVMModule (this=this@entry=0x3f2e3a0, module=module@entry=0x3f861d0, error="", optLevel=optLevel@entry=1)
    at /usr/src/debug/beignet-1.3.2-5.fc29.x86_64/backend/src/backend/program.cpp:127
#14 0x00007ffff70afad6 in gbe::genProgramNewFromLLVM (deviceID=22807, module=0x3f861d0, llvm_ctx=0x3f45750, asm_file_name=<optimized out>, stringSize=1048576, err=0x4033910 "", errSize=0x3f53d10, optLevel=1, 
    options=0x4304fa0 "-I opencl -cl-mad-enable -D__GPU__ -DDEVICE_INFO=34 -D__SIZEOF_HOST_SIZE_T__=8 -DDEV_VER_MAJOR=1 -DDEV_VER_MINOR=3 -D_OPENCL_COMPILER -DKEYLEN=64 -DSALTLEN=64 -DOUTLEN=68")
    at /usr/src/debug/beignet-1.3.2-5.fc29.x86_64/backend/src/backend/gen_program.cpp:488
#15 0x00007ffff6f5de86 in gbe::programNewFromSource (deviceID=22807, source=<optimized out>, stringSize=1048576, 
    options=0x4304fa0 "-I opencl -cl-mad-enable -D__GPU__ -DDEVICE_INFO=34 -D__SIZEOF_HOST_SIZE_T__=8 -DDEV_VER_MAJOR=1 -DDEV_VER_MINOR=3 -D_OPENCL_COMPILER -DKEYLEN=64 -DSALTLEN=64 -DOUTLEN=68", err=0x4033910 "", 
    errSize=0x3f53d10) at /usr/include/c++/8/bits/basic_string.h:1022
#16 0x00007ffff76cd859 in cl_program_build (p=p@entry=0x3f53c00, 
    options=options@entry=0x4304fa0 "-I opencl -cl-mad-enable -D__GPU__ -DDEVICE_INFO=34 -D__SIZEOF_HOST_SIZE_T__=8 -DDEV_VER_MAJOR=1 -DDEV_VER_MINOR=3 -D_OPENCL_COMPILER -DKEYLEN=64 -DSALTLEN=64 -DOUTLEN=68")
    at /usr/src/debug/beignet-1.3.2-5.fc29.x86_64/src/cl_program.c:597
#17 0x00007ffff76c28df in clBuildProgram (program=0x3f53c00, num_devices=<optimized out>, device_list=<optimized out>, 
    options=0x4304fa0 "-I opencl -cl-mad-enable -D__GPU__ -DDEVICE_INFO=34 -D__SIZEOF_HOST_SIZE_T__=8 -DDEV_VER_MAJOR=1 -DDEV_VER_MINOR=3 -D_OPENCL_COMPILER -DKEYLEN=64 -DSALTLEN=64 -DOUTLEN=68", pfn_notify=0x0, user_data=0x0)
    at /usr/src/debug/beignet-1.3.2-5.fc29.x86_64/src/cl_api.c:835
#18 0x00007ffff7b71472 in clBuildProgram (program=0x3f53c00, num_devices=num_devices@entry=0, device_list=device_list@entry=0x0, 
    options=options@entry=0x4304fa0 "-I opencl -cl-mad-enable -D__GPU__ -DDEVICE_INFO=34 -D__SIZEOF_HOST_SIZE_T__=8 -DDEV_VER_MAJOR=1 -DDEV_VER_MINOR=3 -D_OPENCL_COMPILER -DKEYLEN=64 -DSALTLEN=64 -DOUTLEN=68", 
    pfn_notify=pfn_notify@entry=0x0, user_data=user_data@entry=0x0) at ocl_icd_loader_gen.c:387
#19 0x00000000007104fa in opencl_build (sequential_id=sequential_id@entry=0, opts=opts@entry=0x7fffffffc940 "-DKEYLEN=64 -DSALTLEN=64 -DOUTLEN=68", save=save@entry=1, 
    file_name=file_name@entry=0x7fffffffc6f0 "$JOHN/opencl/zip_kernel.cl_1e580caf21d82755d8464c708552feb0.bin", program=program@entry=0x1f85540 <program>, kernel_source_file=0x1fc62c0 "/home/fd/git/john/run/opencl/zip_kernel.cl", 
    kernel_source_file@entry=0x9b4178 "$JOHN/opencl/zip_kernel.cl", 
    kernel_source=0x3fafc30 "/*\n * This software is Copyright (c) 2018 magnum\n * and it is hereby released to the general public under the following terms:\n * Redistribution and use in source and binary forms, with or without\n * "...) at opencl_common.c:1262
#20 0x0000000000712630 in opencl_build_kernel (kernel_filename=kernel_filename@entry=0x9b4178 "$JOHN/opencl/zip_kernel.cl", sequential_id=0, opts=opts@entry=0x7fffffffc940 "-DKEYLEN=64 -DSALTLEN=64 -DOUTLEN=68", warn=warn@entry=0)
    at opencl_common.c:2312
#21 0x00000000007143b9 in opencl_init (kernel_filename=kernel_filename@entry=0x9b4178 "$JOHN/opencl/zip_kernel.cl", sequential_id=<optimized out>, opts=opts@entry=0x7fffffffc940 "-DKEYLEN=64 -DSALTLEN=64 -DOUTLEN=68")
    at opencl_common.c:2381
#22 0x0000000000806512 in reset (db=0x3f39650) at opencl_zip_fmt_plug.c:184
#23 0x000000000069df0c in fmt_self_test_body (full_lvl=<optimized out>, db=0x3f39650, salt_copy=<optimized out>, binary_copy=<optimized out>, format=0xad4300 <fmt_opencl_zip>) at formats.c:832
#24 fmt_self_test (format=format@entry=0xad4300 <fmt_opencl_zip>, db=db@entry=0x3f39650) at formats.c:1938
#25 0x000000000068f52c in benchmark_all () at bench.c:905
#26 0x00000000006a8132 in john_run () at john.c:1642
#27 main (argc=<optimized out>, argv=<optimized out>) at john.c:2053

Plus 33 failing formats:

Device 1: Intel(R) HD Graphics Kabylake Desktop GT1.5
Benchmarking: KeePass-opencl [SHA256 AES/Twofish/ChaCha OpenCL]... FAILED (cmp_all(1))
Benchmarking: TrueCrypt-opencl [RIPEMD160 AES256_XTS OpenCL]... FAILED (cmp_all(1))
Benchmarking: agilekeychain-opencl, 1Password Agile Keychain [PBKDF2-SHA1 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: ansible-opencl, Ansible Vault [PBKDF2-SHA256 HMAC-SHA256 OpenCL]... FAILED (cmp_all(1))
Benchmarking: axcrypt-opencl [SHA1 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: axcrypt2-opencl, AxCrypt 2.x [PBKDF2-SHA512 AES OpenCL]... FAILED (cmp_all(1))
LWS=8 GWS=4096 Benchmarking: bcrypt-opencl ("$2a$05", 32 iterations) [Blowfish OpenCL]... FAILED (cmp_all(1))
Benchmarking: Bitcoin-opencl, Bitcoin Core [SHA512 OpenCL AES 64/64]... (8xOMP) FAILED (cmp_all(4))
Benchmarking: BitLocker-opencl, BitLocker [SHA256 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: blockchain-opencl, blockchain My Wallet (v2 x5000) [PBKDF2-SHA1 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: cloudkeychain-opencl, 1Password Cloud Keychain [PBKDF2-SHA512 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: sha512crypt-opencl, crypt(3) $6$ (rounds=5000) [SHA512 OpenCL]... FAILED (cmp_all(1))
Benchmarking: diskcryptor-opencl, DiskCryptor [PBKDF2-SHA512 OpenCL]... (8xOMP) FAILED (cmp_all(4))
Benchmarking: electrum-modern-opencl, Electrum Wallet 2.8+ [PBKDF2-SHA512 OpenCL]... (8xOMP) FAILED (cmp_all(4))
Benchmarking: ethereum-presale-opencl, Ethereum Presale Wallet [PBKDF2-SHA256 AES Keccak OpenCL]... FAILED (cmp_all(1))
Benchmarking: geli-opencl, FreeBSD GELI [PBKDF2-SHA512 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: gpg-opencl, OpenPGP / GnuPG Secret Key [SHA1/SHA2 OpenCL]... (8xOMP) FAILED (cmp_all(1))
Benchmarking: keychain-opencl, Mac OS X Keychain [PBKDF2-SHA1 3DES OpenCL]... FAILED (cmp_all(1))
Benchmarking: keyring-opencl, GNOME Keyring [SHA256 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: ODF-opencl, OpenDocument Star/Libre/OpenOffice [PBKDF2-SHA1 BF/AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: office-opencl, MS Office [SHA1/SHA512 AES OpenCL]... FAILED (cmp_all(4))
Benchmarking: PBKDF2-HMAC-SHA512-opencl, GRUB2 / OS X 10.8+ [PBKDF2-SHA512 OpenCL]... FAILED (cmp_all(4))
Benchmarking: pgpdisk-opencl, PGP Disk / Virtual Disk [SHA1 AES/TwoFish/CAST OpenCL]... FAILED (cmp_all(1))
Benchmarking: pgpsda-opencl, PGP Self Decrypting Archive [SHA1 CAST OpenCL]... FAILED (cmp_all(1))
Benchmarking: pgpwde-opencl, PGP Whole Disk Encryption [SHA1 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: sappse-opencl, SAP PSE [PKCS#12 PBE (SHA1) OpenCL]... FAILED (cmp_all(1))
Benchmarking: ssh-opencl [RSA/DSA/EC (SSH private keys) OpenCL]... FAILED (cmp_all(1))
Benchmarking: sspr-opencl, NetIQ SSPR / Adobe AEM [MD5/SHA1/SHA2 OpenCL]... FAILED (cmp_all(1))
Benchmarking: strip-opencl, Password Manager [PBKDF2-SHA1 AES OpenCL]... FAILED (cmp_all(1))
Benchmarking: telegram-opencl [PBKDF2-SHA1 AES OpenCL 4x]... FAILED (cmp_all(1))
Benchmarking: wpapsk-opencl, WPA/WPA2/PMF/PMKID PSK [PBKDF2-SHA1 OpenCL]... FAILED (cmp_all(7))
Benchmarking: wpapsk-pmk-opencl, WPA/WPA2/PMF/PMKID master key [MD5/SHA-1/SHA-2 OpenCL]... FAILED (cmp_all(3))
Benchmarking: zed-opencl, Prim'X Zed! encrypted archives [PKCS#12 PBE (SHA1/SHA256) OpenCL]... FAILED (cmp_all(1))
magnumripper commented 3 years ago

That GPU stuff is Beignet though, not POCL, right? Or is it both? No, Beignet is complete on its own I think. What's the output of --list=opencl-devices?

claudioandre-br commented 3 years ago

As I "announced" in https://github.com/openwall/john/issues/4543#issuecomment-757328330, Intel and POCL may be working together right now.

Intent/purpose: Intel OpenCL CPU Runtime integration into POCL

I forgot to announce this. It's old news, but I'm sure people are not aware: https://sourceforge.net/p/pocl/mailman/pocl-devel/thread/1517615B361A5B4797D7811F9FD4F0FE22F23A3B%40IRSMSX107.ger.corp.intel.com/#msg36587306