search

openwall / john

John the Ripper jumbo - advanced offline password cracker, which supports hundreds of hash and cipher types, and runs on many operating systems, CPUs, GPUs, and even some FPGAs
https://www.openwall.com/john/
9.65k stars 2.04k forks source link

JtR on Windows: OpenCL DLL usage #3191

Closed dwendt closed 5 months ago

dwendt commented 6 years ago

This bug initially started with clGetPlatformIDs returning -1001 which is CL_PLATFORM_NOT_FOUND_KHR and john dies which is wrong/bad.

hashcat on the same system detects the platform fine using literally the same function. The only conclusion I can come to is that hashcat is doing something correctly in their build, and john is not.

Steps to reproduce

  1. grab the first build of bleeding-jumbo you can find on google http://openwall.info/wiki/_media/john/johntheripper-v1.8.0.12-jumbo-1-bleeding-e6214ceab-2018-02-07-win-x64.7z
  2. install a graphics card whose platform is opencl 1.2
  3. run john.exe --list=opencl-devices with a card that's restricted to OCL 1.2 platform (this should work fine with 2.1/2.2 headers...)

System configuration

windows enterprise 10.0.16299.19, 64-bit

H:\john180j1w\bleeding>.\run\john.exe --verbosity=5 --list=opencl-devices
Warning: '/dev/shm' does not exists or is not a directory.

POSIX shared memory objects require the existance of this directory.
Create the directory '/dev/shm' and set the permissions to 01777.
For instance on the command line: mkdir -m 01777 /dev/shm
initUnicode(UNICODE, UTF-8/ISO-8859-1)
UTF-8 -> UTF-8 -> UTF-8
Error: No OpenCL-capable platforms were detected by the installed OpenCL driver.
Throw clError: clGetPlatformIDs() = -1001
Error: No OpenCL-capable devices were detected by the installed OpenCL driver.

H:\john180j1w\bleeding>.\run\john.exe --list=build-info
Warning: '/dev/shm' does not exists or is not a directory.

POSIX shared memory objects require the existance of this directory.
Create the directory '/dev/shm' and set the permissions to 01777.
For instance on the command line: mkdir -m 01777 /dev/shm
Version: 1.8.0.12-jumbo-1-bleeding-e6214ceab 2018-02-07 17:56:37 +0100
Build: cygwin 64-bit x86_64 SSE4.2 AC MPI + OMP
SIMD: SSE4.1, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
CPU tests: SSE4.2
$JOHN is ./run/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 6.4.0
OpenCL headers version: 2.2
Crypto library: OpenSSL
OpenSSL library version: 0100020ef
OpenSSL 1.0.2n  7 Dec 2017
GMP library version: 6.1.2
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
H:\hashcat-4.0.1>hashcat64.exe -I
hashcat (v4.0.1) starting...

OpenCL Info:

Platform ID #1
  Vendor  : NVIDIA Corporation
  Name    : NVIDIA CUDA
  Version : OpenCL 1.2 CUDA 9.1.84

  Device ID #1
    Type           : GPU
    Vendor ID      : 32
    Vendor         : NVIDIA Corporation
    Name           : GeForce GTX 970
    Version        : OpenCL 1.2 CUDA
    Processor(s)   : 13
    Clock          : 1253
    Memory         : 1024/4096 MB allocatable
    OpenCL Version : OpenCL C 1.2
    Driver Version : 391.01

root cause

almost working after replacing the junk cygopencl DLL

POSIX shared memory objects require the existance of this directory. Create the directory '/dev/shm' and set the permissions to 01777. For instance on the command line: mkdir -m 01777 /dev/shm initUnicode(UNICODE, UTF-8/ISO-8859-1) UTF-8 -> UTF-8 -> UTF-8 Device 0@konata: GeForce GTX 970 Using default input encoding: UTF-8 Loaded 1 password hash (dmg-opencl, Apple DMG [PBKDF2-SHA1 OpenCL 3DES/AES]) Cost 1 (iteration count) is 270270 for all loaded hashes Will run 8 OpenMP threads Loaded 7 hashes with 7 different salts to test db from test vectors Options used: -I /run/kernels -cl-mad-enable -DSM_MAJOR=5 -DSM_MINOR=2 -cl-nv-verbose -DGPU -DDEVICE_INFO=262162 -DSIZEOF_SIZE_T=8 -DDEV_VER_MAJOR=391 -DDEV_VER_MINOR=1 -D_OPENCL_COMPILER -DHASH_LOOPS=753 -DOUTLEN=32 -DPLAINTEXT_LENGTH=64 -DV_WIDTH=1 $JOHN/kernels/pbkdf2_hmac_sha1_kernel.cl Build log: :46:10: fatal error: 'opencl_device_info.h' file not found

include "opencl_device_info.h"

     ^

Error -11 building kernel $JOHN/kernels/pbkdf2_hmac_sha1_kernel.cl. DEVICE_INFO=262162 OpenCL CL_BUILD_PROGRAM_FAILURE (-11) error in common-opencl.c:1111 - clBuildProgram failed.

dwendt commented 6 years ago

so, there's a pretty funny workaround for the path thing that makes it work 100% correctly with the official OpenCL.dll supplied by driver manufacturers. since only the current directory is getting searched for the opencl headers, just...

  1. cd into ./run/kernels
  2. ../john.exe whatever-arguments-you-want-here
  3. everything should work fine!
claudioandre-br commented 6 years ago

Could you please check if you have a file named c:\Windows\System32\nvopencl.dll ?

claudioandre-br commented 6 years ago

If the nvopencl.dll exists at the specified location (and you are in the mood), could you try this build?

Download and extract win_x64.zip

[1] 7FD1A4618FD3462A7FF3AA03A52F87FDA61FEB1F49BC4A77F1C434DAE7DFE64F

In case you want to see what is that, you can read:

claudioandre-br commented 6 years ago

Waiting news from you. Still, thinking out loud

I do not have (or had) any NVIDIA or Intel hardware. So, I never tested it properly. Anyway, I did test the win_x64.zip package seen above using an AMD GPU on Win 64 bits and I noticed that:

dwendt commented 6 years ago

It does work, out of the box. Was it due to an older version of cygOpenCL-1.dll?

PS H:\john\1.8J1-bc1bbc96f> .\run\john.exe --list=opencl-devices
Platform #0 name: NVIDIA CUDA, version: OpenCL 1.2 CUDA 9.1.84
    Device #0 (0) name:     GeForce GTX 970
    Device vendor:          NVIDIA Corporation
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 CUDA
    Driver version:         391.01 [recommended]
    Native vector widths:   char 1, short 1, int 1, long 1
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          4 GB
    Global Memory Cache:    208 KB
    Local Memory:           48 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 1024 MB
    Max clock (MHz):        1253
    Profiling timer res.:   1000 ns
    Max Work Group Size:    1024
    Parallel compute cores: 13
    CUDA cores:             1664  (13 x 128)
    Speed index:            2084992
    Warp size:              32
    Max. GPRs/work-group:   65536
    Compute capability:     5.2 (sm_52)
    Kernel exec. timeout:   yes
    PCI device topology:    01:00.0

PS H:\john\1.8J1-bc1bbc96f> .\run\john.exe --list=build-info
Version: 1.8.0.12-jumbo-1-bleeding-bc1bbc96f 2018-03-07 07:58:55 -0300
Build: cygwin 64-bit x86_64 AVX AC OMP
SIMD: AVX, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
CPU tests: AVX
CPU fallback binary: john-sse41
OMP fallback binary: john-avx-non-omp
$JOHN is /run/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 6.4.0
OpenCL headers version: 2.2
Crypto library: OpenSSL
OpenSSL library version: 0100020ef      (loaded: 0100020df)
OpenSSL 1.0.2n  7 Dec 2017      (loaded: OpenSSL 1.0.2m  2 Nov 2017)
GMP library version: 6.1.2
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
dwendt commented 6 years ago

Well, there's still strange path problems going on. The previous workaround for that still works, though.

PS H:\john\1.8J1-bc1bbc96f> ./run/john.exe --verbosity=5 --format=dmg-opencl --wordlist=H:\rockyou.txt H:\example.txt
initUnicode(UNICODE, UTF-8/ISO-8859-1)
UTF-8 -> UTF-8 -> UTF-8
Device 0: GeForce GTX 970
Using default input encoding: UTF-8
Loaded 1 password hash (dmg-opencl, Apple DMG [PBKDF2-SHA1 3DES/AES OpenCL])
Cost 1 (iteration count) is 270270 for all loaded hashes
Cost 2 (version) is 2 for all loaded hashes
Loaded 7 hashes with 7 different salts to test db from test vectors
Options used: -I /run/kernels -cl-mad-enable -DSM_MAJOR=5 -DSM_MINOR=2 -cl-nv-verbose -D__GPU__ -DDEVICE_INFO=262162 -DSIZEOF_SIZE_T=8 -DDEV_VER_MAJOR=391 -DDEV_VER_MINOR=1 -D_OPENCL_COMPILER -DHASH_LOOPS=500 -DOUTLEN=32 -DPLAINTEXT_LENGTH=64 -DV_WIDTH=1 $JOHN/kernels/dmg_kernel.cl
Build log: <kernel>:14:10: fatal error: 'pbkdf2_hmac_sha1_kernel.cl' file not found
#include "pbkdf2_hmac_sha1_kernel.cl"
         ^

Error -11 building kernel $JOHN/kernels/dmg_kernel.cl. DEVICE_INFO=262162
OpenCL CL_BUILD_PROGRAM_FAILURE (-11) error in opencl_common.c:1115 - clBuildProgram failed.
claudioandre-br commented 6 years ago

It does work, out of the box. Was it due to an older version of cygOpenCL-1.dll?

No. It happened because cygOpenCL-1.dll needs a config file [1] that points to nvopencl.dll [2].

Well, there's still strange path problems going on. The previous workaround for that still works, though.

But, have you replaced cygOpenCL-1.dll with OpenCL.dll?

[1] the ICD vendor file. [2] well, since I don't have NVIDIA, I asked you to check if the file really exists, or if my (cof, Google) guess was wrong.

dwendt commented 6 years ago

Nope, the DLL isn't replaced. This is just regularly unzipping that build to a folder, and running that cmdline.

I guess my last posts weren't too clear: nvopencl.dll and OpenCL.dll exist in C:\Windows\System32 -- both are shipped with nvidia's drivers. They seem unrelated to john being unable to find the kernel include path. (procmon shows john searches the current working directory for the header -- and nowhere else)

claudioandre-br commented 6 years ago

Ok. We need to think about how to fix it.

claudioandre-br commented 6 years ago

[edited] Ops, it was the 32bits binary link. Sorry.

Could you please try https://ci.appveyor.com/project/claudioandre/johntheripper/build/1.8J1-688d3d200/job/s7m1q7pic9smx6i5/artifacts Important:

Just an example (run it the way you used to)

C:\bleeding>del \bleeding\run\kernels\*.bin && \bleeding\run\john --test=0 --format=sha512crypt-opencl
Device 0: Juniper [AMD Radeon HD 6700 Series]
Testing: sha512crypt-opencl, crypt(3) $6$ (rounds=5000) [SHA512 OpenCL]... ---
Testing: ./run/kernels
Final: ./run/kernels
Where: /
PASS

[1] In fact, for NVIDIA, the cache will be:

AppVeyor interface is nice. It is easy to find the latest build, get 32 or 64 bits binaries, etc.

claudioandre-br commented 6 years ago

Ops, the link I used was for the 32-bit binary. Excuse me.

Fixed.

arcfide commented 6 years ago

I'm having trouble using OpenCL on Windows with stuff like this. I'm confused reading the thread above what the correct process is to getting this to work? I've read about ICD vendor files for Cygwin, and I think I understand what I need to put in the ICD file to get things to work, but I can't see where I would put the ICD vendor file. Where in the JtR unzipped program folder should I put an ICD vendor file?

kholia commented 6 years ago

Where in the JtR unzipped program folder should I put an ICD vendor file?

This thread doesn't talk about ICD vendor file(s) at all, if I am reading things correctly.

I will be ("soon" -> no real eta) working on making sure that JtR's OpenCL support just runs on Windows without any hard hacks.

kholia commented 6 years ago

@claudioandre-br Hey, do you already have some AppVeyor configuration (appveyor.yml) to build JtR Jumbo using Cygwin on Windows? If yes, could you please open a PR to include this build configuration in our official repository?

This way, I can build on top of your work and don't have to start from scratch. Thanks!

arcfide commented 6 years ago

Okay, I got this fixed up. If you see claudioandre-br's comment, that's where ICD Vendor files are mentioned. He also gives a working example of a build that seems to work. I've got this working now on the current build.

It doesn't require any hard hacks, but I did figure out that the OpenCL drivers with Cygwin don't work without an ICD Vendor file. That means that there has to be a location to find such files. That means that the OpenCL support works on Windows if you run JtR from inside of a Cygwin installation.

To make this work for me on Windows, I installed Cygwin with OpenCL, and then created the /etc/OpenCL/vendors/nvidia.icd file that included the Cygwin path to nvopencl.dll mentioned above. After I did that, I ran JtR from inside of the Cygwin Terminal, which mounts and makes available the /etc/ directory. That has fixed things, and I can now see all of my devices and I can run JtR on the GPU with the appropriate speedups.

claudioandre-br commented 6 years ago

OpenCL drivers with Cygwin don't work without an ICD Vendor file.

Indeed

To make this work for me on Windows, I installed Cygwin with OpenCL [...] I ran JtR from inside of the Cygwin Terminal

It it possible to run it without Cygwin installed. Using this zip file (https://rebrand.ly/JtRWin64), you can run JtR on a regular Windows machine (GPU drivers required, or course). And it is 100% magnum ripper source code.

kholia commented 6 years ago

Thanks for these hints.

I am looking forward to create redistributable JtR-Jumbo-OpenCL builds which just work without requiring installation of a Cygwin environment. Maybe it is not possible to do what I am planning to do? I don't know yet.

kholia commented 6 years ago

@claudioandre-br Hey, do you already have some AppVeyor configuration (appveyor.yml) to build JtR Jumbo using Cygwin on Windows? If yes, could you please open a PR to include this build configuration in our official repository?

I see that you have https://github.com/claudioandre-br/packages/blob/master/john-the-ripper/tests/appveyor.yml.

Could it be made part of the official JtR Jumbo repository? Can we get the latest Windows + Cygwin builds to test using this method? If yes, this will greatly reduce a lot of friction for the end-users.

claudioandre-br commented 6 years ago

do you already have some AppVeyor configuration (appveyor.yml) to build JtR

Yes, but it was designed to test, not exactly to create a package.

redistributable JtR-Jumbo-OpenCL builds which just work without requiring installation of a Cygwin environment. Maybe it is not possible to do what I am planning to do?

Yes, it is. It did this in https://rebrand.ly/JtRWin64 In my process, I'm building it as a special since my process was designed to test JtR:

Could it be made part of the official JtR Jumbo repository?

Yes. But I created to file to test. It might not be suitable to be used as you intend.

claudioandre-br commented 6 years ago

Just in case anyone arrives here by chance, an introductory text:

I'm closing the bug. It was not reporting a bug.

kholia commented 6 years ago

Could it be made part of the official JtR Jumbo repository?

Yes. But I created this file to test. It might not be suitable to be used as you intend.

I see. Nevertheless, it would be great to include this file in the official source tree. It currently does JtR Jumbo testing and with some future work, we can probably even generate ready-to-use builds using this AppVeyor configuration file.

https://github.com/claudioandre-br/packages/blob/master/john-the-ripper/readme.md#windows

This file contains great information :+1: Some of the content here belongs in the official source tree.

claudioandre-br commented 6 years ago

Anything can be copied and used. Including instructions and the https://rebrand.ly/JtRWin64 link.

I'm not going to do this myself because a user can report as an issue:

Hey magnum

It installed JtR using the link https://rebrand.ly/JtRWin64. Two weeks later, I accessed 
www.OMGosh-photos.com and my computer was hacked. What should I do?

I bet magnum will say that packaging is something OpenWall should do.

kholia commented 5 years ago

Thanks. I can do a few PR(s) regarding this - hopefully not too far away in the future.

We already build, test, and provide Win32 MinGW builds using the official repository. These builds aren't perfect (one issue -> AVX2 missing on 64-bit builds due to GCC bug) but they are better than nothing.

With some more work (and more documentation), these MinGW builds can be made even more usable.

With your work on Cygwin builds, we will have even more usable and featureful builds in the near future :+1:

All this needs to be part of the official source tree, else it will bitrot sooner or later.

@solardiz @magnumripper I would like to hear your opinions on this topic of creating usable builds for end-users. Would doing so be a desirable thing?

I know that @magnumripper is concerned about us having to provide support for such builds but if we set the right expectations with our users (one idea -> GitHub issue template will ask users to post to john-users regarding support for any binary builds instead of opening a issue in the repository), this shouldn't be a big problem. It will be a win for us (as developers) as we would get to iron out a few bugs in our code base, I imagine.

magnumripper commented 5 years ago

I bet magnum will say that packaging is something OpenWall should do.

Well, either that or no-one. I certainly wont.

magnumripper commented 5 years ago

@solardiz @magnumripper I would like to hear your opinions on this topic of creating usable builds for end-users. Would doing so be a desirable thing?

Not to me... but perhaps the end users would love it. As long as no-one gets the idea to sue the balls off me for something I didn't do, I don't mind.

solardiz commented 5 years ago

@solardiz @magnumripper I would like to hear your opinions on this topic of creating usable builds for end-users. Would doing so be a desirable thing?

@kholia Of course, we need usable builds for end-users. We need this not only to help the users, but also to keep our project known and relevant, which means bug reports (I mean including for real bugs in the code), code contributions, etc. If everyone on Windows only uses hashcat or mdxfind or whatever else there might be, that brings JtR into irrelevance not only for those Windows users, but to a smaller extent also for the rest of prospective users - many of them would happen to read about those other tools and not about JtR, and wouldn't consider JtR even if their platform is better supported in JtR.

technowhiz commented 5 years ago

Hi, I am a newbie to john the ripper. I am getting the opencl error which is specified here. could anyone explain me how to replace the cygOpencl-1.dll with openCL.dll in the john the ripper directory. I copied the OpenCL.dll from system32 folder to the 'run' directory inside the john the ripper directory but got the same opencl error. then i renamed the OpenCL.dll to cygOpenCL-1.dll. now i am getting cmd error "unable to start application 0xc000007b". kindly help me to solve this issue. thanks in advance

Edit: dropped the extreme over-quoting.

solardiz commented 5 years ago

@sdk156 We use GitHub issues to track things for us to fix in JtR, not to provide user support. Your question would be appropriate for the john-users mailing list: https://www.openwall.com/lists/john-users/

Then, when you use the latest Windows build currently on JtR homepage (of version 1.9.0-jumbo-1), the problem shouldn't occur. You didn't mention what version you have the problem with (you should; and if it's not the latest, then we won't care).

Finally, please avoid over-quoting. Instead of the quote, you should have provided actual error output you're getting.

Summary: start by trying out the latest version/build off JtR homepage. If you still have issues with it, join the john-users mailing list and post in there (and mention what version/build of JtR you try to use). Thanks!

solardiz commented 5 years ago

@sdk156 You should also mention the type of your GPU, what GPU drivers you installed (if any), and Windows version and bitness.

Reviewing past comments on this issue, I see @arcfide mentioning that things work with Cygwin's DLL when ICD files are added. That's similar to what we did in our currently released builds, where we included ICD files with typical paths to NVIDIA and AMD DLLs. (But not to Intel initially - that's fixed in Claudio's later builds.)

technowhiz commented 5 years ago

@solardiz my aplologies for the incomplete error report. I have a NVIDIA Geforce MX150 GPU with latest NVIDIA drivers installed. And I was getting the same error as listed below. Error: No OpenCL-capable platforms were detected by the installed OpenCL driver. Error: No OpenCL-capable devices were detected by the installed OpenCL driver.

the releases i was trying as follows,

  1. winX64_1_JtR.7z - latest dev version
  2. john-1.9.0-jumbo-1-win64 - latest windows binary from openwall/john 3.johntheripper-v1.8.0.12-jumbo-1-bleeding-e6214ceab-2018-02-07-win-x64.7z - as suggested in this thread 4.x64_win.zip

and fortunately, i have resolved the issue by replacing the contents of nvidia.icd under 'etc/OpenCL/vendors'. The default contents of the file is 'c:\Windows\System32\nvopencl.dll'. I have replaced this with the actual path of nvopencl64.dll in another path under system32 folder. Now jon the ripper successfully listed my gpu when using the command 'john --list=opencl-devices'

solardiz commented 5 years ago

Thank you for this additional detail, @sdk156.

What was the actual path of nvopencl64.dll on your system? Do you know why it's different from the default?

I wonder if we can somehow list multiple/alternate paths, @claudioandre-br.

technowhiz commented 5 years ago

@solardiz , sorry . I am a noob to opencl environments. In this thread , I have read that nvopencl.dll might be found under system32 folder. so i just searched it inside system32 but it was not under system32 root but under a subdirectory as follows 'C:\Windows\System32\DriverStore\FileRepository\nvdmi.inf_amd64_b67df4d626c9e7b6'. so i replaced this path instead of the contents of nvidia.icd in 'etc/OpenCL/vendors'. And it worked. Once again sorry, i dont know why i have a different path for nvopencl64.dll. thank you.

solardiz commented 5 years ago

We appreciate the detailed info you provided, @sdk156. Thank you!

technowhiz commented 5 years ago

@solardiz welcome and thank you for your appreciation :) :) :)

samyH commented 5 years ago

Hi guys,

I'm having issues getting my GPU (Nvidia GTX 860M) to even output as a device when running ./john --list=opencl-devices.

The driver I currently have installed (via Device Manager) is 23.21.13.8813 - I have tried rolling back to drivers from 2014 and also updating to 2019 driver with no luck.

I am using Cygwin64 with john-1.9.0-jumbo-1-win64 build from openwall website.

I also tried changing the path inside the nvidia.icd vendor file to the location of nvopencl64.dll as mentioned above, but no luck.

I know it's not the fastest GPU in the world, but hopefully, it'll help me get to my password a little quicker if I can get this working correctly.

Thanks in advance.

solardiz commented 5 years ago

Dear all, let me inform/remind you that our use of GitHub is to track issues for us to fix. It is not to provide user support. We have the john-users mailing list for that. So if you post something worded as a support request on our GitHub, you're doing it wrong. ;-)

I also tried changing the path inside the nvidia.icd vendor file to the location of nvopencl64.dll as mentioned above, but no luck.

@samyH Did you find where nvopencl64.dll is located on your system and try using that path? There's almost no chance @sdk156's cryptic path would also be right for your system.

technowhiz commented 5 years ago

@samyH , I think there is no need to use cygwin in windows to run JTR. And i read in one of the forums that, windows still does not support gpu access via its windows subsytem for linux (WSL). plz follow this WSL gpu issue. this issue is still open. Try replacing the path of nvopencl64.dll with your own and try running JTR in command prompt.

solardiz commented 5 years ago

@claudioandre-br @alainesp Can we possibly extract the vendors' OpenCL backend DLL pathnames from somewhere in Windows and populate (e.g., from JtR startup code) the .icd file(s) for Cygwin's cygOpenCL-1.dll that our builds use? Where is this information normally stored on Windows - e.g., registry or does it have .icd file(s) somewhere?

alainesp commented 5 years ago

Hi Solar.

On Wed, Jul 3, 2019 at 7:46 AM Solar Designer notifications@github.com wrote:

@claudioandre-br https://github.com/claudioandre-br @alainesp https://github.com/alainesp Can we possibly extract the vendors' OpenCL backend DLL pathnames from somewhere in Windows and populate the .icd file(s) for Cygwin's cygOpenCL-1.dll that our builds use? Where is this information normally stored on Windows - e.g., registry or does it have .icd file(s) somewhere?

This project may help https://github.com/KhronosGroup/OpenCL-ICD-Loader In the Readme.md they write:

On Windows, add the "stub" ICD by adding a REG_DWORD value to the registry keys:

// For 32-bit operating systems, or 64-bit tests on a 64-bit operating system: HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\OpenCL\Vendors

// For 32-bit tests on a 64-bit operating system: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Khronos\OpenCL\Vendors

// The name of the REG_DWORD value should be the full path to the "stub" ICD // OpenCLDriverStub.dll, and the data for this value should be 0.

Hope this helps, Alain

solardiz commented 5 years ago

Thanks Alain. So what we may do is one of:

  1. Read those registry keys and populate an .icd file for use by cygOpenCL-1.dll.

  2. Load Windows' native OpenCL.dll from our code.

  3. Can we just link to Windows' native OpenCL.dll instead of to Cygwin's somehow, so that we don't have to complicate our code with manual loading of the library?

  4. As a last resort, maybe we can include a script or whatever that would do what Claudio suggests here:

https://github.com/claudioandre-br/packages/blob/master/john-the-ripper/readme.md#windows

"the first advice to be given to anyone facing Windows problems would be: replacing cygwin's OpenCL library cygOpenCL-1.dll with OpenCL.dll installed in the System32 folder should make everything almost work."

If that's in fact our advice, then we should make it available to users and easy to follow, not have it "hidden" in Claudio's repo.

I'll reopen the issue now. Claudio closed it with "I'm closing the bug. It was not reporting a bug." Bug or not, this is an issue that many people keep running into, and we ought to do something about that.

solardiz commented 5 years ago

The original issue description here says:

somehow, this hashcat release made hashcat prefer the library load path DLL over cygOpenCL-1.dll: https://hashcat.net/forum/thread-6187-post-32962.html

I just found the corresponding hashcat commit: https://github.com/hashcat/hashcat/commit/33aeae60908e9dcdc9165dff5897d90d661ca9f6

It does basically:

   #elif defined (__CYGWIN__)
-  ocl->lib = hc_dlopen ("cygOpenCL-1.dll", RTLD_NOW);
+  ocl->lib = hc_dlopen ("opencl.dll", RTLD_NOW);
+
+  if (ocl->lib == NULL) ocl->lib = hc_dlopen ("cygOpenCL-1.dll", RTLD_NOW);

So hashcat was already using dlopen before that point. We don't yet.

solardiz commented 5 years ago

@dwendt Maybe you'd want to look into this issue further and contribute a fix to our project in the form of a pull request? We'd appreciate that! Most of us are not on Windows, but it appears that you are, and it appears that you'd also be capable of coming up with a proper fix for all, not just for your computer.

dwendt commented 5 years ago

Hey @solardiz - apologies for opening this and then orphaning it. I was working through this for a side project, totally unrelated to my day job, and I see that other people have apparently had the same issue...

Thanks for finding that change - I'm going to get john building and experiment on a fresh Windows install on both an AMD & Nvidia system to see if doing something similar solves it. I'll PR some time this weekend if it does.

StillMyGitHub commented 4 years ago

I had the same issue with john-1.9.0-jumbo-1-win64 and Radeon RX 580. replacing the dll and cd-ing to \kernels\ did not work for me at first try, the funny thing I noticed is, that this does only work with relative adressing of john.exe. If you use absolute adressing (C:\Users\%Username%....\john.exe) john cannot find the kernel headers even if it is started from that path.

Furthermore: After a first successful run which builds the kernel (a .bin file in the same kernels folder) you can run john (for that hashtype) from any directory

I wonder whether... a) it is possible to just precompile all kernels b) how customized/ portable the OpenCL kernels are c) why it makes a difference wether you start john with absolute or relative path

magnumripper commented 4 years ago

a) Pre-compiling is not possible right now. We might want to have that, especially for DEScrypt per-salt kernels (4K of them...).

b) It depends. Some vendors' binary kernels are tightly coupled to runtime/driver version and hardware. I believe some others aren't, they're more like a compile-as-far-as-you-can-without-getting-specific. I'm sure @solardiz knows more about it.

c) That's a jolly good question. I'm not sure I even want to hear the answer 🤣

claudioandre-br commented 4 years ago

c) why it makes a difference wether you start john with absolute or relative path

c) That's a jolly good question. I'm not sure I even want to hear the answer

I was aware of that. When you run JtR from outside the CygWin root folder, it fails to resolve its "home" properly.

magnumripper commented 4 years ago

So what can we do about that? Isn't there a good system call to use on Windows crap?

solardiz commented 4 years ago

On Wed, Oct 09, 2019 at 04:36:59PM -0700, Claudio Andr?? wrote:

When you run JtR from outside the CygWin root folder, it fails to resolve its "home" properly.

  • the CygWin function call that retrieves the path fails.

If so, perhaps this affects more than just OpenCL kernels - e.g., does it also affect JtR's ability to find its john.conf and the CPU fallback binaries? If somehow this issue does not affect those things (why not?) then perhaps we can revise our OpenCL support code to also avoid the issue in the same way as those other places do?

claudioandre-br commented 4 years ago

I can't remember. I would say:

Some Windows tests have to be run.

claudioandre-br commented 4 years ago

For the record ([1]):

My expectation was that a proprietary GPU driver installation on Windows would add OpenCL "instructions" to the registry at _HKEY_LOCALMACHINE\SOFTWARE\Khronos\OpenCL\Vendors. Unfortunately, this is not a rule [2].

On my machine I did not find any OpenCL significant information within the registry. To make OpenCL work I did:

I do not believe that there is a universal (mine, for example) Openll.dll that works for everyone.

That said, #2133 is a good idea for 2.0 series.

[1] I just purchased a new Acer "Ryzen" notebook. [2[ The AMD GPU/OpenCL came installed from the "factory" (OEM).