openwall / john

John the Ripper jumbo - advanced offline password cracker, which supports hundreds of hash and cipher types, and runs on many operating systems, CPUs, GPUs, and even some FPGAs
https://www.openwall.com/john/
Other
9.98k stars 2.06k forks source link

Opencl and GPU's #4167

Closed davidwarr closed 4 years ago

davidwarr commented 4 years ago

This a request and issue, I keep getting this error,

modprobe: FATAL: Module fglrx not found in directory /lib/modules/5.2.0-kali2-amd64 Error! Fail to load fglrx kernel module! Maybe you can switch to root user to load kernel module directly

I can not find fglrx anywhere at least not anything that will work. I researched it and found out it was discontinued in 2015. Then re-released as Radeon Software Crimson Edition in version 15.11, after that it was changed to Adrenalin edition. Simple enough, no JTR wont accept anything other than fglrx it wont use the intel GPU because of the AMD GPU. I dug around inside the JTR files and found opencl it hasn't been updated since 2017. src/opencl/wpapsk_kernel.cl

john --list=opencl-devices modprobe: FATAL: Module fglrx not found in directory /lib/modules/5.2.0-kali2-amd64 Error! Fail to load fglrx kernel module! Maybe you can switch to root user to load kernel module directly Platform #0 name: Portable Computing Language, version: OpenCL 1.2 pocl 1.3 None+Asserts, LLVM 8.0.1, SLEEF, DISTRO, POCL_DEBUG Device #0 (1) name: pthread-Intel(R) Core(TM) i7-3770k CPU @ 3.40GHz Device vendor: GenuineIntel Device type: CPU (LE) Device version: OpenCL 1.2 pocl HSTR: pthread-x86_64-pc-linux-gnu-ivybridge Driver version: 1.3 Native vector widths: char 16, short 16, int 8, long 4 Preferred vector width: char 16, short 16, int 8, long 4 Global Memory: 21931 MB Global Memory Cache: 8 MB Local Memory: 4 MB (Global) Constant Buffer size: 4 MB Max memory alloc. size: 8 GB Max clock (MHz): 3900 Profiling timer res.: 1 ns Max Work Group Size: 4096 Parallel compute cores: 8 Speed index: 124800

Platform #1 name: Intel(R) CPU Runtime for OpenCL(TM) Applications, version: OpenCL 2.1 LINUX Device #0 (2) name: Intel(R) Core(TM) i7-3770k CPU @ 3.40GHz Device vendor: Intel(R) Corporation Device type: CPU (LE) Device version: OpenCL 2.1 (Build 0) Driver version: 18.1.0.0920 Native vector widths: char 16, short 8, int 4, long 2 Preferred vector width: char 1, short 1, int 1, long 1 Global Memory: 23979 MB Global Memory Cache: 256 KB Local Memory: 32 KB (Global) Constant Buffer size: 128 KB Max memory alloc. size: 5994 MB Max clock (MHz): 3400 Profiling timer res.: 1 ns Max Work Group Size: 8192 Parallel compute cores: 8 Speed index: 54400

Platform #2 name: AMD Accelerated Parallel Processing, version: OpenCL 1.2 AMD-APP (1113.2) Device #0 (3) name: Intel(R) Core(TM) i7-3770k CPU @ 3.40GHz Device vendor: GenuineIntel Device type: CPU (LE) Device version: OpenCL 1.2 AMD-APP (1113.2) Driver version: 1113.2 (sse2,avx) Native vector widths: char 16, short 8, int 4, long 2 Preferred vector width: char 16, short 8, int 4, long 2 Global Memory: 23979 MB Global Memory Cache: 32 KB Local Memory: 32 KB (Global) Constant Buffer size: 64 KB Max memory alloc. size: 5994 MB Max clock (MHz): 3691 Profiling timer res.: 1 ns Max Work Group Size: 1024

john --format=opencl /root/JTR/xxx-JTR modprobe: FATAL: Module fglrx not found in directory /lib/modules/5.2.0-kali2-amd64 Error! Fail to load fglrx kernel module! Maybe you can switch to root user to load kernel module directly Device 1: pthread-Intel(R) Core(TM) i7-3770k CPU @ 3.40GHz Warning: detected hash type "wpapsk-opencl", but the string is also recognized as "wpapsk-pmk-opencl" Use the "--format=wpapsk-pmk-opencl" option to force loading these as that type instead Using default input encoding: UTF-8 Loaded 1 password hash (wpapsk-opencl, WPA/WPA2/PMF/PMKID PSK [PBKDF2-SHA1 OpenCL]) s

^CSession aborted

If you don't abort the session nothing happens.

john /root/JTR/xxx-JTR Warning: detected hash type "wpapsk", but the string is also recognized as "wpapsk-pmk" Use the "--format=wpapsk-pmk" option to force loading these as that type instead Warning: detected hash type "wpapsk", but the string is also recognized as "wpapsk-opencl" Use the "--format=wpapsk-opencl" option to force loading these as that type instead Warning: detected hash type "wpapsk", but the string is also recognized as "wpapsk-pmk-opencl" Use the "--format=wpapsk-pmk-opencl" option to force loading these as that type instead Using default input encoding: UTF-8 Loaded 1 password hash (wpapsk, WPA/WPA2/PMF/PMKID PSK [PBKDF2-SHA1 128/128 AVX 4x]) Will run 8 OpenMP threads

Then is runs very slowly around 1000 per p/s, c/s and C/s If I remove the GPU or purge then reinstall and don't use opencl its around 6000 per p/s, c/s and C/s. In Windows it is about the same 6000 or there abouts. I'm reporting as a bug because it needs to be brought up to date. The opencl and that's as far as I looked is two years old I don't know if that matters 100% but fglrx is four years past end of life, (About to be five.) I don't know if anyone will agree with me since it is running correctly circa 2015 if the age of the opencl file doesn't pertain to the situation. If it isn't something that is going to be fixed might I suggest a statement saying JTR wont work with those cards. Had I known and I would have purchased a Nividia, which wouldn't matter anyway if support for Cuda is really being terminated. I did look before I bought it from what I found it said JTR worked with AMD. If I might sway you as to an update. I was reading on google a few days ago that AMD CPUs are preferred by 60% of Europeans. Usually as goes the Europeans so goes the rest of the world. Although the TechSpot news piece says CPU I would imagine they may have the same problems as the GPU.

claudioandre-br commented 4 years ago

fglrx is the proprietary graphics driver for OLDER AMD/ATI graphic cards.

The development of fglrx has stopped. AMD continues to support the development of the open source radeon driver module and the newer AMDGPU driver, with AMDGPU-PRO as the proprietary closed source extension.

So, fglrx will not be updated for use with newer versions of X.org Xserver, as well as Wayland support won't be added.


Are you sure your GPU and distro require and support fglrx?

claudioandre-br commented 4 years ago

JTR wont accept anything other than fglrx it wont use the intel GPU because of the AMD GPU.

[edited]

If it isn't something that is going to be fixed might I suggest a statement saying JTR wont work with those cards.

JtR works fine in fglrx (I use it inside CI), in AMDGPU, in NVIDIA and Intel GPUs (we have these in Openwall's servers). For example:

$ $JtR --list=opencl-devices
Platform #0 name: AMD Accelerated Parallel Processing, version: OpenCL 2.0 AMD-APP (1642.5)
    Device #0 (1) name:     Tahiti
    Board name:             AMD Radeon HD 7900 Series  
    Device vendor:          Advanced Micro Devices, Inc.
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 AMD-APP (1642.5)
    Driver version:         1642.5 (VM) - Catalyst 14.12(Omega) [supported]
    Native vector widths:   char 4, short 2, int 1, long 1
    Preferred vector width: char 4, short 2, int 1, long 1
    Global Memory:          3013 MB
    Global Memory Cache:    16 KB
    Local Memory:           32 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 2097 MB
    Max clock (MHz):        1000
    Profiling timer res.:   1 ns
    Max Work Group Size:    256
    Parallel compute cores: 32
    Stream processors:      2048  (32 x 64)
    Speed index:            2048000
    SIMD width:             16
    Wavefront width:        64
    ADL:                    Overdrive5, device id 0
    PCI device topology:    03:00.0
    PCI lanes:              16/16
    Fan speed:              20%
    Temperature:            49C
    Utilization:            7%

    Device #1 (2) name:     Tahiti
    Board name:             AMD Radeon HD 7900 Series  
    Device vendor:          Advanced Micro Devices, Inc.
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 AMD-APP (1642.5)
    Driver version:         1642.5 (VM) - Catalyst 14.12(Omega) [supported]
    Native vector widths:   char 4, short 2, int 1, long 1
    Preferred vector width: char 4, short 2, int 1, long 1
    Global Memory:          3014 MB
    Global Memory Cache:    16 KB
    Local Memory:           32 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 2100 MB
    Max clock (MHz):        1000
    Profiling timer res.:   1 ns
    Max Work Group Size:    256
    Parallel compute cores: 32
    Stream processors:      2048  (32 x 64)
    Speed index:            2048000
    SIMD width:             16
    Wavefront width:        64
    ADL:                    Overdrive32664, device id -1
    PCI device topology:    03:00.0

    Device #2 (3) name:     Bonaire
    Board name:             AMD Radeon HD 7700 Series
    Device vendor:          Advanced Micro Devices, Inc.
    Device type:            GPU (LE)
    Device version:         OpenCL 2.0 AMD-APP (1642.5)
    Driver version:         1642.5 (VM) - Catalyst 14.12(Omega) [supported]
    Native vector widths:   char 4, short 2, int 1, long 1
    Preferred vector width: char 4, short 2, int 1, long 1
    Global Memory:          1988 MB
    Global Memory Cache:    16 KB
    Local Memory:           32 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 1333 MB
    Max clock (MHz):        1075
    Profiling timer res.:   1 ns
    Max Work Group Size:    256
    Parallel compute cores: 14
    Stream processors:      896  (14 x 64)
    Speed index:            963200
    SIMD width:             16
    Wavefront width:        64
    ADL:                    Overdrive5, device id 0
    PCI device topology:    03:00.0
    PCI lanes:              16/16
    Fan speed:              20%
    Temperature:            49C
    Utilization:            0%

    Device #3 (4) name:     Juniper
    Board name:             AMD Radeon HD 6700 Green Edition
    Device vendor:          Advanced Micro Devices, Inc.
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 AMD-APP (1642.5)
    Driver version:         1642.5 - Catalyst 14.12(Omega) [supported]
    Native vector widths:   char 16, short 8, int 4, long 2
    Preferred vector width: char 16, short 8, int 4, long 2
    Global Memory:          512 MB
    Local Memory:           32 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 128 MB
    Max clock (MHz):        700
    Profiling timer res.:   1 ns
    Max Work Group Size:    256
    Parallel compute cores: 9
    Stream processors:      720  (9 x 80)
    Speed index:            504000
    SIMD width:             16
    Wavefront width:        64
    ADL:                    Overdrive32664, device id -1
    PCI device topology:    03:00.0

    Device #4 (5) name:     Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
    Device vendor:          GenuineIntel
    Device type:            CPU (LE)
    Device version:         OpenCL 1.2 AMD-APP (1642.5)
    Driver version:         1642.5 (sse2,avx) 
    Native vector widths:   char 16, short 8, int 4, long 2
    Preferred vector width: char 16, short 8, int 4, long 2
    Global Memory:          31902 MB
    Global Memory Cache:    32 KB
    Local Memory:           32 KB (Global)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 7975 MB
    Max clock (MHz):        3800
    Profiling timer res.:   1 ns
    Max Work Group Size:    1024
    Parallel compute cores: 8
    Speed index:            60800

Platform #1 name: Intel(R) OpenCL, version: OpenCL 1.2 
    Device #0 (6) name:     Intel(R) HD Graphics
    Device vendor:          Intel(R) Corporation
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 
    Driver version:         16.4.2.1.39163 
    Native vector widths:   char 1, short 1, int 1, long 1
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          1630 MB
    Global Memory Cache:    256 KB
    Local Memory:           64 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 407 MB
    Max clock (MHz):        1250
    Profiling timer res.:   80 ns
    Max Work Group Size:    512
    Parallel compute cores: 20
    Stream processors:      160  (20 x 8)
    Speed index:            200000

    Device #1 (7) name:     Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
    Device vendor:          Intel(R) Corporation
    Device type:            CPU (LE)
    Device version:         OpenCL 1.2 (Build 9756)
    Driver version:         1.2.0.9756 
    Native vector widths:   char 32, short 16, int 8, long 4
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          31902 MB
    Global Memory Cache:    256 KB
    Local Memory:           32 KB (Global)
    Constant Buffer size:   128 KB
    Max memory alloc. size: 7975 MB
    Max clock (MHz):        3500
    Profiling timer res.:   1 ns
    Max Work Group Size:    8192
    Parallel compute cores: 8
    Speed index:            112000

[edited] NVIDIA plus new AMD boards

$ ../run/john -list=opencl-devices
Platform #0 name: AMD Accelerated Parallel Processing, version: OpenCL 2.1 AMD-APP (2766.4)
    Device #0 (1) name:     gfx900
    Board name:             Radeon RX Vega
    Device vendor:          Advanced Micro Devices, Inc.
    Device type:            GPU (LE)
    Device version:         OpenCL 2.0 AMD-APP (2766.4)
    Driver version:         2766.4 (PAL,HSAIL) - AMDGPU-Pro  
    Native vector widths:   char 4, short 2, int 1, long 1
    Preferred vector width: char 4, short 2, int 1, long 1
    Global Memory:          8176 MB
    Global Memory Cache:    16 KB
    Local Memory:           64 KB (Local)
    Constant Buffer size:   4048 MB
    Max memory alloc. size: 4048 MB
    Max clock (MHz):        1107
    Profiling timer res.:   1 ns
    Max Work Group Size:    256
    Parallel compute cores: 64
    Stream processors:      4096  (64 x 64)
    Speed index:            4534272
    SIMD width:             16
    Wavefront width:        64
    PCI device topology:    05:00.0

Platform #1 name: Intel(R) OpenCL, version: OpenCL 1.2 LINUX
    Device #0 (2) name:     Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
    Device vendor:          Intel(R) Corporation
    Device type:            CPU (LE)
    Device version:         OpenCL 1.2 (Build 76921)
    Driver version:         1.2.0.76921 
    Native vector widths:   char 16, short 8, int 4, long 2
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          126 GB
    Global Memory Cache:    256 KB
    Local Memory:           32 KB (Global)
    Constant Buffer size:   128 KB
    Max memory alloc. size: 32291 MB
    Max clock (MHz):        2000
    Profiling timer res.:   1 ns
    Max Work Group Size:    8192
    Parallel compute cores: 32
    Speed index:            128000

    Device #1 (3) name:     Intel(R) Many Integrated Core Acceleration Card
    Device vendor:          Intel(R) Corporation
    Device type:            Accelerator (LE)
    Device version:         OpenCL 1.2 (Build 76921)
    Driver version:         1.2 
    Native vector widths:   char 16, short 16, int 16, long 8
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          5773 MB (ECC)
    Global Memory Cache:    256 KB
    Local Memory:           32 KB (Global)
    Constant Buffer size:   128 KB
    Max memory alloc. size: 1924 MB
    Max clock (MHz):        1052
    Profiling timer res.:   1052 ns
    Max Work Group Size:    8192
    Parallel compute cores: 236
    Stream processors:      1888  (236 x 8)
    Speed index:            1986176

Platform #2 name: NVIDIA CUDA, version: OpenCL 1.2 CUDA 10.1.105
    Device #0 (4) name:     GeForce GTX 1080
    Device vendor:          NVIDIA Corporation
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 CUDA
    Driver version:         418.39 [recommended]
    Native vector widths:   char 1, short 1, int 1, long 1
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          8119 MB
    Global Memory Cache:    320 KB
    Local Memory:           48 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 2029 MB
    Max clock (MHz):        1733
    Profiling timer res.:   1000 ns
    Max Work Group Size:    1024
    Parallel compute cores: 20
    CUDA cores:             2560  (20 x 128)
    Speed index:            4436480
    Warp size:              32
    Max. GPRs/work-group:   65536
    Compute capability:     6.1 (sm_61)
    Kernel exec. timeout:   no
    NVML id:                0
    PCI device topology:    01:00.0
    PCI lanes:              8/16
    Fan speed:              0%
    Temperature:            59°C
    Utilization:            0%

    Device #1 (5) name:     GeForce GTX TITAN X
    Device vendor:          NVIDIA Corporation
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 CUDA
    Driver version:         418.39 [recommended]
    Native vector widths:   char 1, short 1, int 1, long 1
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          12212 MB
    Global Memory Cache:    384 KB
    Local Memory:           48 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 3053 MB
    Max clock (MHz):        1076
    Profiling timer res.:   1000 ns
    Max Work Group Size:    1024
    Parallel compute cores: 24
    CUDA cores:             3072  (24 x 128)
    Speed index:            3305472
    Warp size:              32
    Max. GPRs/work-group:   65536
    Compute capability:     5.2 (sm_52)
    Kernel exec. timeout:   no
    NVML id:                2
    PCI device topology:    84:00.0
    PCI lanes:              16/16
    Fan speed:              53%
    Temperature:            83°C
    Utilization:            100%

    Device #2 (6) name:     GeForce GTX TITAN
    Device vendor:          NVIDIA Corporation
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 CUDA
    Driver version:         418.39 [recommended]
    Native vector widths:   char 1, short 1, int 1, long 1
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          6083 MB
    Global Memory Cache:    224 KB
    Local Memory:           48 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 1520 MB
    Max clock (MHz):        954
    Profiling timer res.:   1000 ns
    Max Work Group Size:    1024
    Parallel compute cores: 14
    CUDA cores:             2688  (14 x 192)
    Speed index:            2564352
    Warp size:              32
    Max. GPRs/work-group:   65536
    Compute capability:     3.5 (sm_35)
    Kernel exec. timeout:   no
    NVML id:                1
    PCI device topology:    83:00.0
    PCI lanes:              16/16
    Fan speed:              35%
    Temperature:            53°C
    Utilization:            0%
claudioandre-br commented 4 years ago

Notice:

claudioandre-br commented 4 years ago

Although this issue is worded as a bug report, not as a question, I'm afraid you're mistaken.

We assume you are genuinely thinking you are reporting a real bug to help the project. That said, please:

claudioandre-br commented 4 years ago

Example of the fglrx error message [1]:

[1] No valid/compatible GPU was found. Reasoning:

davidwarr commented 4 years ago

Thank you for your explanation. I'm not getting the correct installation info for fglrx according to what you showed me. I'll keep trying I'm using the pro drivers. I ran the tests from your link they seem to be the same as in the link. I'll rebuild and retry on Ubuntu. I do have a question about that if it's ok. Are you suggesting I use Ubuntu 14.04, I will that's not a problem I just wanted to make sure I was correct.

john -test-full=0 --format=cpu Will run 8 OpenMP threads Testing: descrypt, traditional crypt(3) [DES 128/128 AVX]... (8xOMP) PASS Testing: bsdicrypt, BSDI crypt(3) ("_J9..", 725 iterations) [DES 128/128 AVX]... (8xOMP) PASS Testing: md5crypt, crypt(3) $1$ (and variants) [MD5 128/128 AVX 4x3]... (8xOMP) PASS Testing: md5crypt-long, crypt(3) $1$ (and variants) [MD5 32/64]... (8xOMP) PASS Testing: bcrypt ("$2a$05", 32 iterations) [Blowfish 32/64 X3]... (8xOMP) PASS Testing: scrypt (16384, 8, 1) [Salsa20/8 128/128 AVX]... (8xOMP) PASS Testing: LM [DES 128/128 AVX]... (8xOMP) PASS Testing: AFS, Kerberos AFS [DES 48/64 4K]... PASS Testing: tripcode [DES 128/128 AVX]... (8xOMP) PASS Testing: AndroidBackup [PBKDF2-SHA1 128/128 AVX 4x AES]... (8xOMP) PASS Testing: adxcrypt [IBM/Toshiba 4690 - ADXCRYPT 32/64]... (8xOMP) PASS Testing: agilekeychain, 1Password Agile Keychain [PBKDF2-SHA1 AES 128/128 AVX 4x]... (8xOMP) PASS john -test-full=0 --format=opencl modprobe: FATAL: Module fglrx not found in directory /lib/modules/5.2.0-kali2-amd64 Error! Fail to load fglrx kernel module! Maybe you can switch to root user to load kernel module directly Device 1: pthread-Intel(R) Core(TM) i7-3770k CPU @ 3.40GHz Testing: sha1crypt-opencl, (NetBSD) [PBKDF1-SHA1 OpenCL 8x]... PASS Testing: KeePass-opencl [SHA256 AES/Twofish/ChaCha OpenCL]... PASS Testing: oldoffice-opencl, MS Office <= 2003 [MD5/SHA1 RC4 OpenCL]... PASS Testing: PBKDF2-HMAC-MD4-opencl [PBKDF2-MD4 OpenCL 8x]... PASS Testing: PBKDF2-HMAC-MD5-opencl [PBKDF2-MD5 OpenCL 8x]... PASS Testing: PBKDF2-HMAC-SHA1-opencl [PBKDF2-SHA1 OpenCL 8x]... PASS Testing: rar-opencl, RAR3 (length 5) [SHA1 OpenCL AES]... (8xOMP) PASS

It is passing the tests I saw on your link.

Then

john --format=wpapsk-OpenCL /root/JTR/xxx-JTR modprobe: FATAL: Module fglrx not found in directory /lib/modules/5.2.0-kali2-amd64 Error! Fail to load fglrx kernel module! Maybe you can switch to root user to load kernel module directly