Various problems with Mesa

[devurandom]

x86_64-pc-linux-gnu-gcc jumbo.o john-mpi.o DES_fmt.o DES_std.o DES_bs.o DES_bs_b.o BSDI_fmt.o MD5_fmt.o MD5_std.o cryptmd5_common.o BF_fmt.o BF_std.o BF_common.o scrypt_fmt.o escrypt/crypto_scrypt-best.o escrypt/crypto_scrypt-common.o e
scrypt/sha256.o AFS_fmt.o LM_fmt.o trip_fmt.o NT_fmt.o timer.o md5.o rc4.o hmacmd5.o base64.o base64_convert.o md4.o sha2.o dynamic_fmt.o dynamic_parser.o dynamic_preloads.o dynamic_utils.o dynamic_big_crypt.o ripemd.o tiger.o ssh2john.
o pfx2john.o unrarcmd.o unrarfilter.o unrarhlp.o unrar.o unrarppm.o unrarvm.o rar2john.o zip2john.o pkzip.o gladman_hmac.o gladman_pwd2key.o racf2john.o keychain2john.o kwallet2john.o pwsafe2john.o dmg2john.o keepass2john.o keyring2john
.o hccap2john.o 7z_fmt_plug.o BFEgg_fmt_plug.o DMD5_fmt_plug.o DOMINOSEC8_fmt_plug.o DOMINOSEC_fmt_plug.o EPI_fmt_plug.o FGT_fmt_plug.o HDAA_fmt_plug.o IPB2_fmt_plug.o KRB4_fmt_plug.o KRB4_std_plug.o KRB5_fmt_plug.o KRB5_std_plug.o Kecc
akDuplex_plug.o KeccakF-1600-opt64_plug.o KeccakHash_plug.o KeccakSponge_plug.o MSCHAPv2_bs_fmt_plug.o NETLM_fmt_plug.o NETLMv2_fmt_plug.o NETNTLM_bs_fmt_plug.o NETNTLMv2_fmt_plug.o NETSPLITLM_fmt_plug.o NS_fmt_plug.o PHPS2_fmt_plug.o P
HPS_fmt_plug.o PO_fmt_plug.o SKEY_fmt_plug.o SKEY_jtr_plug.o SybaseASE_fmt_plug.o SybasePROP_fmt_plug.o XSHA512_fmt_plug.o XSHA_fmt_plug.o agilekeychain_fmt_plug.o aix_smd5_fmt_plug.o aix_ssha_fmt_plug.o androidfde_fmt_plug.o asaMD5_fmt
_plug.o bcrypt_pbkdf_plug.o bitcoin_fmt_plug.o blackberry_ES10_fmt_plug.o blake2b-ref_plug.o blake2b_plug.o blf_plug.o blockchain_fmt_plug.o chap_fmt_plug.o citrix_ns_fmt_plug.o clipperz_srp_fmt_plug.o cloudkeychain_fmt_plug.o common_op
encl_pbkdf2_plug.o cq_fmt_plug.o crc32_fmt_plug.o crypt-sha1_fmt_plug.o cryptsha256_fmt_plug.o cryptsha512_fmt_plug.o cuda_cryptmd5_fmt_plug.o cuda_cryptsha256_fmt_plug.o cuda_cryptsha512_fmt_plug.o cuda_mscash2_fmt_plug.o cuda_mscash_f
mt_plug.o cuda_phpass_fmt_plug.o cuda_pwsafe_fmt_plug.o cuda_rawsha512_fmt_plug.o cuda_wpapsk_fmt_plug.o cuda_xsha512_fmt_plug.o dahua_fmt_plug.o django_fmt_plug.o django_scrypt_fmt_plug.o dmg_fmt_plug.o dragonfly3_fmt_plug.o dragonfly4
_fmt_plug.o drupal7_fmt_plug.o dynamic_compiler_fmt_plug.o dynamic_compiler_plug.o ecryptfs_fmt_plug.o efs_fmt_plug.o eigrp_fmt_plug.o encfs_common_plug.o encfs_fmt_plug.o episerver_fmt_plug.o feal8_plug.o formspring_fmt_plug.o gost3411
-2012-sse41_plug.o gost3411-tables_plug.o gost_fmt_plug.o gpg_fmt_plug.o has160_fmt_plug.o has160_plug.o haval_fmt_plug.o haval_plug.o hmacMD5_fmt_plug.o hmacSHA1_fmt_plug.o hmacSHA224_fmt_plug.o hmacSHA256_fmt_plug.o hmacSHA384_fmt_plug.o hmacSHA512_fmt_plug.o hmailserver_fmt_plug.o hsrp_fmt_plug.o idea_plug.o ike_fmt_plug.o keepass_fmt_plug.o keychain_fmt_plug.o keyring_fmt_plug.o keystore_fmt_plug.o known_hosts_fmt_plug.o krb5-18_fmt_plug.o krb5pa-md5_fmt_plug.o krb5pa-sha1_fmt_plug.o kwallet_fmt_plug.o lastpass_fmt_plug.o lastpass_sniffed_fmt_plug.o lotus5_fmt_plug.o lotus85_fmt_plug.o luks_fmt_plug.o md2_fmt_plug.o md2_plug.o md4_gen_fmt_plug.o mdc2_fmt_plug.o mdc2dgst_plug.o mediawiki_fmt_plug
.o mongodb_fmt_plug.o mozilla_ng_fmt_plug.o mscash1_fmt_plug.o mscash2_fmt_plug.o mssql-old_fmt_plug.o mssql05_fmt_plug.o mssql12_fmt_plug.o mysqlSHA1_fmt_plug.o mysql_fmt_plug.o mysql_netauth_fmt_plug.o net_md5_fmt_plug.o net_sha1_fmt_plug.o nsldap_fmt_plug.o nt2_fmt_plug.o ntlmv1_mschapv2_fmt_plug.o nukedclan_fmt_plug.o o5logon_fmt_plug.o odf_fmt_plug.o office_common_plug.o office_fmt_plug.o oldoffice_fmt_plug.o openbsdsoftraid_fmt_plug.o opencl_7z_fmt_plug.o opencl
_DES_bs_b_plug.o opencl_DES_bs_f_plug.o opencl_DES_bs_h_plug.o opencl_DES_bs_plug.o opencl_DES_fmt_plug.o opencl_agilekeychain_fmt_plug.o opencl_bf_fmt_plug.o opencl_bf_std_plug.o opencl_blockchain_fmt_plug.o opencl_cryptmd5_fmt_plug.o 
opencl_cryptsha256_fmt_plug.o opencl_cryptsha512_fmt_plug.o opencl_dmg_fmt_plug.o opencl_encfs_fmt_plug.o opencl_gpg_fmt_plug.o opencl_keychain_fmt_plug.o opencl_keyring_fmt_plug.o opencl_krb5pa-md5_fmt_plug.o opencl_krb5pa-sha1_fmt_plu
g.o opencl_lotus5_fmt_plug.o opencl_mscash2_fmt_plug.o opencl_mysqlsha1_fmt_plug.o opencl_nsldaps_fmt_plug.o opencl_nt_fmt_plug.o opencl_ntlmv2_fmt_plug.o opencl_o5logon_fmt_plug.o opencl_odf_aes_fmt_plug.o opencl_odf_fmt_plug.o opencl_
office2007_fmt_plug.o opencl_office2010_fmt_plug.o opencl_office2013_fmt_plug.o opencl_oldoffice_fmt_plug.o opencl_pbkdf2_hmac_md4_fmt_plug.o opencl_pbkdf2_hmac_md5_fmt_plug.o opencl_pbkdf2_hmac_sha1_fmt_plug.o opencl_pbkdf2_hmac_sha256
_fmt_plug.o opencl_pbkdf2_hmac_sha512_fmt_plug.o opencl_phpass_fmt_plug.o opencl_pwsafe_fmt_plug.o opencl_rakp_fmt_plug.o opencl_rar5_fmt_plug.o opencl_rar_fmt_plug.o opencl_rawmd4_fmt_plug.o opencl_rawmd5_fmt_plug.o opencl_rawsha1_fmt_
plug.o opencl_rawsha256_fmt_plug.o opencl_rawsha512_fmt_plug.o opencl_sha1crypt_fmt_plug.o opencl_sha2_common_plug.o opencl_strip_fmt_plug.o opencl_sxc_fmt_plug.o opencl_wpapsk_fmt_plug.o opencl_zip_fmt_plug.o openssl_enc_fmt_plug.o ora
cle11_fmt_plug.o oracle12c_fmt_plug.o oracle_fmt_plug.o osc_fmt_plug.o panama_fmt_plug.o panama_plug.o pbkdf2-hmac-md4_fmt_plug.o pbkdf2-hmac-md5_fmt_plug.o pbkdf2-hmac-sha1_fmt_plug.o pbkdf2-hmac-sha512_fmt_plug.o pbkdf2_hmac_sha256_fm
t_plug.o pdf_fmt_plug.o pdfcrack_md5_plug.o pfx_fmt_plug.o phpassMD5_fmt_plug.o pixMD5_fmt_plug.o pkzip_fmt_plug.o plaintext_fmt_plug.o pomelo_fmt_plug.o pomelo_plug.o postgres_fmt_plug.o pst_fmt_plug.o pufferfish_api_plug.o pufferfish_
fmt_plug.o pufferfish_itoa64_plug.o pufferfish_plug.o putty_fmt_plug.o pwsafe_fmt_plug.o racf_fmt_plug.o radmin_fmt_plug.o rakp_fmt_plug.o rar5_fmt_plug.o rar_fmt_plug.o rawBLAKE2_512_fmt_plug.o rawKeccak_256_fmt_plug.o rawKeccak_512_fm
t_plug.o rawMD4_fmt_plug.o rawMD5_fmt_plug.o rawMD5flat_fmt_plug.o rawSHA0_fmt_plug.o rawSHA1_fmt_plug.o rawSHA1_linkedIn_fmt_plug.o rawSHA1_ng_fmt_plug.o rawSHA224_fmt_plug.o rawSHA256_common_plug.o rawSHA256_fmt_plug.o rawSHA256_ng_fm
t_plug.o rawSHA384_fmt_plug.o rawSHA512_common_plug.o rawSHA512_fmt_plug.o rawSHA512_ng_fmt_plug.o rawmd5u_fmt_plug.o ripemd_fmt_plug.o rsvp_fmt_plug.o salted_sha1_fmt_plug.o sapB_fmt_plug.o sapG_fmt_plug.o sapH_fmt_plug.o sha1_gen_fmt_
plug.o sha1crypt_common_plug.o sha3_512_fmt_plug.o siemens-s7_fmt_plug.o sip_fmt_plug.o skein_fmt_plug.o skein_plug.o snefru_fmt_plug.o snefru_plug.o ssh_fmt_plug.o ssh_ng_fmt_plug.o ssha512_fmt_plug.o stribog_fmt_plug.o strip_fmt_plug.
o sunmd5_fmt_plug.o sxc_fmt_plug.o syb-prop_repro_plug.o tcp_md5_fmt_plug.o tiger_fmt_plug.o truecrypt_fmt_plug.o twofish_plug.o vms_fmt_plug.o vnc_fmt_plug.o vtp_fmt_plug.o wbb3_fmt_plug.o whirlpool_fmt_plug.o whirlpool_plug.o wow_srp_
fmt_plug.o wpapsk_fmt_plug.o zip_fmt_plug.o dyna_salt.o dummy.o gost.o common-gpu.o batch.o bench.o build_table.o charset.o common.o compiler.o config.o cracker.o crc32.o external.o formats.o getopt.o hash_type_128.o hash_type_192.o idl
e.o inc.o john.o list.o loader.o logger.o mask.o mask_ext.o math.o memory.o misc.o options.o params.o path.o recovery.o rpp.o rules.o signals.o single.o status.o tty.o twister.o wordlist.o mkv.o mkvlib.o listconf.o fake_salts.o win32_me
mmap.o uaf_encode.o unicode.o unshadow.o unafs.o undrop.o regex.o pp.o c3_fmt.o unique.o putty2john.o gpg2john.o memdbg.o common-opencl.o opencl_autotune.o x86-64.o sse-intrinsics.o -g -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu   -lss
l -lcrypto  -fopenmp -lgmp    -lOpenCL  -lm -pthread  -lm -lz -ldl -lcrypt  -lbz2 aes/aes.a -o ../run/john
opencl_rawmd4_fmt_plug.o: In function `crypt_all':
opencl_rawmd4_fmt_plug.c:(.text+0x2e33): undefined reference to `clEnqueueFillBuffer'
opencl_rawmd4_fmt_plug.c:(.text+0x2e95): undefined reference to `clEnqueueFillBuffer'
opencl_rawmd5_fmt_plug.o: In function `crypt_all':
opencl_rawmd5_fmt_plug.c:(.text+0x2e33): undefined reference to `clEnqueueFillBuffer'
opencl_rawmd5_fmt_plug.c:(.text+0x2e95): undefined reference to `clEnqueueFillBuffer'
opencl_rawsha1_fmt_plug.o: In function `crypt_all':
opencl_rawsha1_fmt_plug.c:(.text+0x2ba3): undefined reference to `clEnqueueFillBuffer'
opencl_rawsha1_fmt_plug.o:opencl_rawsha1_fmt_plug.c:(.text+0x2c05): more undefined references to `clEnqueueFillBuffer' follow
collect2: error: ld returned 1 exit status

(xz-compressed build.log: The .png extension is just a disguise to fool GitHub - rename the file to build.log.xz after downloading it.)

[magnumripper]

Is this an OpenCL 1.1 driver (as opposed to 1.2)?

[magnumripper]

@sayantan this is just a dry-run guess of mine

diff --git a/src/opencl_rawmd4_fmt_plug.c b/src/opencl_rawmd4_fmt_plug.c
index 4d8e6e6..576ce37 100644
--- a/src/opencl_rawmd4_fmt_plug.c
+++ b/src/opencl_rawmd4_fmt_plug.c
@@ -736,11 +736,13 @@ static int crypt_all(int *pcount, struct db_salt *salt)
        if (!is_static_gpu_mask)
                HANDLE_CLERROR(clEnqueueWriteBuffer(queue[gpu_id], buffer_int_key_loc, CL_TRUE, 0, 4 * global_work_size, saved_int_key_loc, 0, NULL, multi_profilingEvent[2]), "failed in clEnqueueWriteBuffer buffer_int_key_loc.");

+#if CL_VERSION_1_2
        if (ocl_ver >= 120) {
                zero = 0;
                HANDLE_CLERROR(clEnqueueFillBuffer(queue[gpu_id], buffer_hash_ids, &zero, sizeof(cl_uint), 0, sizeof(cl_uint), 0, NULL, multi_profilingEvent[3]), "failed in clEnqueueFillBuffer buffer_hash_ids.");
                HANDLE_CLERROR(clEnqueueFillBuffer(queue[gpu_id], buffer_bitmap_dupe, &zero, sizeof(cl_uint), 0, sizeof(cl_uint) * (hash_table_size/32 + 1), 0, NULL, multi_profilingEvent[4]), "failed in clEnqueueFillBuffer buffer_bitmap_dupe.");
        }
+#endif

        if (salt != NULL && salt->count > 100 &&
                (num_loaded_hashes - num_loaded_hashes / 10) > salt->count) {

Similar fix for all three.

[devurandom]

@magnumripper Yes, as far I know, Mesa supports OpenCL 1.1 only. See e.g. the Gallium Compute feature matrix or various posts on Phoronix.

[magnumripper]

Hopefully fixed in 59c5b42, please try it out and report back.

On a side note, the __OPENCL_VERSION__ macro can not be used (I presume it's only available in device code). But CL_VERSION_1_2 works fine.

[devurandom]

Sorry, still does not work, as Mesa advertises OpenCL 1.2 in its headers, declares the function, but does not define it:

# grep -B1 CL_VERSION /usr/include/CL/cl.h
/* OpenCL Version */
#define CL_VERSION_1_0                              1
#define CL_VERSION_1_1                              1
#define CL_VERSION_1_2                              1

# grep -B1 -A8 clEnqueueFillBuffer /usr/include/CL/cl.h
extern CL_API_ENTRY cl_int CL_API_CALL
clEnqueueFillBuffer(cl_command_queue   /* command_queue */,
                    cl_mem             /* buffer */, 
                    const void *       /* pattern */, 
                    size_t             /* pattern_size */, 
                    size_t             /* offset */, 
                    size_t             /* size */, 
                    cl_uint            /* num_events_in_wait_list */, 
                    const cl_event *   /* event_wait_list */, 
                    cl_event *         /* event */) CL_API_SUFFIX__VERSION_1_2;

# scanelf -s clEnqueueFillBuffer /usr/lib/libOpenCL.so
 TYPE   SYM FILE 
ET_DYN  -  /usr/lib/libOpenCL.so 

Does OpenCL define a way to check the implementation vendor or version at compile time? Maybe that could be used to hack-around this issue?

[magnumripper]

Does OpenCL define a way to check the implementation vendor or version at compile time?

Not that I know of. You should try to find some tell-tale macro or something.

[devurandom]

I reported a bug against Mesa: https://bugs.freedesktop.org/show_bug.cgi?id=91130

[magnumripper]

Try gcc -dM -E -x c /usr/include/CL/cl.h | grep -i mesa (and try replacing 'mesa' with some other guesses) and see if there is some macro (that is not present in gcc -dM -E -x c /dev/null) we can use.

[magnumripper]

As a last resort we could test this function's existence in the configure script.

[devurandom]

$ gcc -dM -E -x c /usr/include/CL/cl.h | grep -i vendor
#define CL_PLATFORM_VENDOR 0x0903
#define CL_DEVICE_VENDOR 0x102C
#define CL_DEVICE_VENDOR_ID 0x1001
$ gcc -dM -E -x c /usr/include/CL/cl.h | grep -i version
#define CL_EXT_SUFFIX__VERSION_1_0_DEPRECATED __attribute__((deprecated))
#define CL_PLATFORM_VERSION 0x0901
#define __GXX_ABI_VERSION 1002
#define CL_DRIVER_VERSION 0x102D
#define __VERSION__ "4.9.2"
#define CL_EXT_PREFIX__VERSION_1_0_DEPRECATED 
#define CL_EXT_SUFFIX__VERSION_1_1_DEPRECATED __attribute__((deprecated))
#define CL_VERSION_1_0 1
#define CL_VERSION_1_1 1
#define CL_VERSION_1_2 1
#define CL_DEVICE_OPENCL_C_VERSION 0x103D
#define CL_DEVICE_VERSION 0x102F
#define CL_EXT_SUFFIX__VERSION_1_0 
#define CL_EXT_SUFFIX__VERSION_1_1 
#define CL_EXT_SUFFIX__VERSION_1_2 
#define CL_API_SUFFIX__VERSION_1_0 
#define CL_API_SUFFIX__VERSION_1_1 
#define CL_API_SUFFIX__VERSION_1_2 
#define CL_EXT_PREFIX__VERSION_1_1_DEPRECATED 
$ gcc -dM -E -x c /usr/include/CL/cl.h | grep -i driver
#define CL_DRIVER_VERSION 0x102D
$ gcc -dM -E -x c /usr/include/CL/cl.h | grep -i platform
#define CL_PLATFORM_VENDOR 0x0903
#define CL_CONTEXT_PLATFORM 0x1084
#define CL_INVALID_PLATFORM -32
#define CL_PLATFORM_VERSION 0x0901
#define CL_DEVICE_PLATFORM 0x1031
#define CL_PLATFORM_NAME 0x0902
#define CL_PLATFORM_PROFILE 0x0900
#define __CL_PLATFORM_H 
#define CL_PLATFORM_EXTENSIONS 0x0904

[magnumripper]

Unfortunately I get these on OSX Yosemite as well. I think Mesa just copied Khronos' headers (which is sensible but they should have used 1.1 instead of 1.2)

$ gcc -dM -E -x c /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/OpenCL.framework/Versions/A/Headers/cl.h | grep -i vendor
#define CL_PLATFORM_VENDOR 0x0903
#define CL_DEVICE_VENDOR 0x102C
#define CL_DEVICE_VENDOR_ID 0x1001

$ gcc -dM -E -x c /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/OpenCL.framework/Versions/A/Headers/cl.h | grep -i driver
#define CL_DRIVER_VERSION 0x102D

$ gcc -dM -E -x c /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/OpenCL.framework/Versions/A/Headers/cl.h | grep -i device_version
#define CL_DEVICE_VERSION 0x102F

$ gcc -dM -E -x c /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/OpenCL.framework/Versions/A/Headers/cl.h | grep -i platform
#define CL_PLATFORM_VENDOR 0x0903
#define CL_CONTEXT_PLATFORM 0x1084
#define CL_PLATFORM_PROFILE 0x0900
#define CL_PLATFORM_VERSION 0x0901
#define CL_DEVICE_PLATFORM 0x1031
#define CL_PLATFORM_NAME 0x0902
#define CL_INVALID_PLATFORM -32
#define __CL_PLATFORM_H 
#define CL_PLATFORM_EXTENSIONS 0x0904

[devurandom]

Did you change something related to this recently?

In any case: d8cb9ce98acd26bd917dff4cbb54bdc14b7133f9 compiles with Mesa 10.6.1.

[devurandom]

I tested using john --test=0 --verbosity=5, but that managed to crash X (and JTR): https://bugs.freedesktop.org/show_bug.cgi?id=91305 (which is not related to this bug, just FYI).

[magnumripper]

Yes I believe Sayantan stopped using those OpenCL 1.2 functions for now. So I guess we're good for now? Please report other problems as separate issues. Did you see what format was running when all crashed?

[devurandom]

Did you see what format was running when all crashed?

Sure, it's all in the logfiles attached to the bug:

Testing: sha512crypt-opencl, crypt(3) $6$ (rounds=5000) [SHA512 OpenCL]... Local worksize (LWS) 7, global worksize (GWS) 49
FAILED (cmp_all(1))

Options used: -I /usr/share/john/kernels -cl-mad-enable -D__GPU__ -DDEVICE_INFO=10 -DDEV_VER_MAJOR=10 -DDEV_VER_MINOR=6 -D_OPENCL_COMPILER
Build log: input.cl:95:43: warning: unknown attribute 'max_constant_size' ignored
input.cl:99:43: warning: unknown attribute 'max_constant_size' ignored

Options used: -I /usr/share/john/kernels -cl-mad-enable -D__GPU__ -DDEVICE_INFO=10 -DDEV_VER_MAJOR=10 -DDEV_VER_MINOR=6 -D_OPENCL_COMPILER
Local worksize (LWS) 8, Global worksize (GWS) 56
Testing: descrypt-opencl, traditional crypt(3) [DES OpenCL]... radeon: Failed to allocate virtual address for buffer:
radeon:    size      : 4 bytes
radeon:    alignment : 4096 bytes
radeon:    domains   : 2
radeon:    va        : 0x00000000023d6000
radeon: Failed to allocate virtual address for buffer:
radeon:    size      : 4 bytes
radeon:    alignment : 4096 bytes
radeon:    domains   : 2
radeon:    va        : 0x0000000001fcb000
FAILED (cmp_one(0))

radeon: Failed to allocate virtual address for buffer:
radeon:    size      : 1 bytes
radeon:    alignment : 1 bytes
radeon:    domains   : 2
radeon:    va        : 0x0000000001fcb000
radeon: Failed to allocate virtual address for buffer:
radeon:    size      : 1 bytes
radeon:    alignment : 1 bytes
radeon:    domains   : 2
radeon:    va        : 0x0000000000858000
Segmentation fault

[magnumripper]

Ahaa, @Sayantan2048 this may be another problem with goto in descrypt-opencl (just a guess though).

@devurandom you could try this change and see if it makes it safer

diff --git a/src/opencl_DES_kernel_params.h b/src/opencl_DES_kernel_params.h
index 1acd8b9..d1586e8 100644
--- a/src/opencl_DES_kernel_params.h
+++ b/src/opencl_DES_kernel_params.h
@@ -14,14 +14,7 @@ typedef unsigned WORD vtype;
  * OSX' Intel HD4000 driver [1.2(Sep25 2014 22:26:04)] fails building the
  * "fast goto" version.
  */
-#if nvidia_sm_5x(DEVICE_INFO) || gpu_intel(DEVICE_INFO) ||       \
-       (gpu_amd(DEVICE_INFO) && DEV_VER_MAJOR >= 1573 && !defined(__Tahiti__)) || \
-       (gpu_amd(DEVICE_INFO) && DEV_VER_MAJOR >= 1702)
-//#warning Using 'safe goto' kernel
 #define SAFE_GOTO
-#else
-//#warning Using 'fast goto' kernel
-#endif

 #if no_byte_addressable(DEVICE_INFO)
 #define RV7xx

[devurandom]

There is a formatting issue on the first line removed: "||\", which is difficult to detect when copy&pasting the patch.

Now it applies and I'll report back with the results shortly.

[devurandom]

Your patch gets me a bit further (running /usr/sbin/john --test=0 --verbosity=5 as a regular non-root user):

Testing: mscash-opencl, M$ Cache Hash [MD4 OpenCL]... Options used: -I /usr/share/john/kernels -cl-mad-enable -D__GPU__ -DDEVICE_INFO=10 -DDEV_VER_MAJOR=10 -DDEV_VER_MINOR=6 -D_OPENCL_COMPILER -D NUM_INT_KEYS=1 -D IS_STATIC_GPU_MASK=0 -D CONST_CACHE_SIZE=268435456 -D LOC_0=-1 -D LOC_1=-1 -D LOC_2=-1 -D LOC_3=-1
Build log: input.cl:365:18: warning: unknown attribute 'max_constant_size' ignored
input.cl:375:18: warning: unknown attribute 'max_constant_size' ignored

Self test GWS: 64, LWS: 8
FAILED (cmp_all(0))

Options used: -I /usr/share/john/kernels -cl-mad-enable -D__GPU__ -DDEVICE_INFO=10 -DDEV_VER_MAJOR=10 -DDEV_VER_MINOR=6 -D_OPENCL_COMPILER -D SALT_BUFFER_SIZE=260
Build log: input.cl:586:17: warning: unknown attribute 'max_constant_size' ignored

binary size 107038
Error creating binary file $JOHN/kernels/pbkdf2_kernel_-D_SALT_BUFFER_SIZE=260_-DDEV_VER_MAJOR=10_-DDEV_VER_MINOR=6AMD_KAVERI_0.bin
Device 0  GWS: 8192, LWS: 8
Testing: mscash2-opencl, MS Cache Hash 2 (DCC2) [PBKDF2-SHA1 OpenCL]... radeon: The kernel rejected CS, see dmesg for more information.
radeon: The kernel rejected CS, see dmesg for more information.
radeon: The kernel rejected CS, see dmesg for more information.
radeon: The kernel rejected CS, see dmesg for more information.
radeon: The kernel rejected CS, see dmesg for more information.
radeon: The kernel rejected CS, see dmesg for more information.
radeon: The kernel rejected CS, see dmesg for more information.
radeon: Failed to allocate virtual address for buffer:
radeon:    size      : 131072 bytes
radeon:    alignment : 4096 bytes
radeon:    domains   : 2
radeon:    va        : 0x00000000020b2000
radeon: Failed to allocate virtual address for buffer:
radeon:    size      : 131072 bytes
radeon:    alignment : 4096 bytes
radeon:    domains   : 2
radeon:    va        : 0x00000000020b2000
Bus error

At this point X crashed again. After this, without rebooting, john will immediately segfault:

$ /usr/sbin/john --test=0 --verbosity=5
radeon: Failed to allocate virtual address for buffer:
radeon:    size      : 4352 bytes
radeon:    alignment : 4096 bytes
radeon:    domains   : 4
radeon:    va        : 0x0000000000800000
radeon: Failed to allocate virtual address for buffer:
radeon:    size      : 4352 bytes
radeon:    alignment : 4096 bytes
radeon:    domains   : 4
radeon:    va        : 0x0000000000800000
Segmentation fault

For the full details, please see https://bugs.freedesktop.org/show_bug.cgi?id=91315.

[magnumripper]

Thanks. We should apply a (proper) patch for descrypt to start with. Would you please post the output from --list=opencl-devices on this system?

[devurandom]

I'll report back next week, when I have access to the machine again. One info already, though: I recently updated to Mesa 10.6.1.

[magnumripper]

It's mscash2-opencl crashing now. Also by @Sayantan2048 (which doesn't imply he's code is bad, it's probably just that he's better at squeezing out performance, which is more prone to reveal driver bugs).

[devurandom]

# john --list=opencl-devices
Invalid MIT-MAGIC-COOKIE-1 keyInvalid MIT-MAGIC-COOKIE-1 keyInvalid MIT-MAGIC-COOKIE-1 keyInvalid MIT-MAGIC-COOKIE-1 keyPlatform #0 name: Clover
Platform version: OpenCL 1.1 MESA 10.6.1
        Device #0 (0) name:     AMD KAVERI
        Device vendor:          AMD
        Device type:            GPU (LE)
        Device version:         OpenCL 1.1 MESA 10.6.1
        Driver version:         10.6.1 - Catalyst 
        Native vector widths:   char 16, short 8, int 4, long 2
        Preferred vector width: char 16, short 8, int 4, long 2
        Global Memory:          1024.0 MB
        Local Memory:           32.0 KB (Local)
        Max memory alloc. size: 256.2 MB
        Max clock (MHz):        720
        Max Work Group Size:    256
        Parallel compute cores: 8
        Stream processors:      640  (8 x 80)

        Device #1 (1) name:     AMD REDWOOD
        Device vendor:          AMD
        Device type:            GPU (LE)
        Device version:         OpenCL 1.1 MESA 10.6.1
        Driver version:         10.6.1 - Catalyst 
        Native vector widths:   char 16, short 8, int 4, long 2
        Preferred vector width: char 16, short 8, int 4, long 2
        Global Memory:          1024.0 MB
        Local Memory:           32.0 KB (Local)
        Max memory alloc. size: 256.2 MB
        Max clock (MHz):        700
        Max Work Group Size:    256
        Parallel compute cores: 5
        Stream processors:      400  (5 x 80)
        PCI device topology:

[magnumripper]

Great, we can look for "MESA" in the device version string. I will look at fixing descrypt.

[magnumripper]

descrypt should be fixed in 8b53844, and we now have our own __MESA__ macro to use for Mesa-specific workarounds. Please confirm that latest source runs descrypt fine as-is.

BTW other changes happened to many formats too, you might want to re-test everything.

[devurandom]

New issue:

opencl_lm_b_plug.o: In function `set_kernel_args_gws':
opencl_lm_b_plug.c:(.text+0x15ea): undefined reference to `clGetKernelArgInfo'
opencl_lm_b_plug.o: In function `set_kernel_args.isra.1':
opencl_lm_b_plug.c:(.text+0x1dc5): undefined reference to `clGetKernelArgInfo'
opencl_lm_b_plug.c:(.text+0x1e4a): undefined reference to `clGetKernelArgInfo'
collect2: error: ld returned 1 exit status

[magnumripper]

This is again caused by Mesa incorrectly defining CL_VERSION_1_2

[magnumripper]

Please retry with latest commit. We now check the actual library for OpenCL 1.2 and don't just trust the headers. Mesa need to correct this though, there will be many problems - just not with JtR Jumbo.

[devurandom]

Sorry, I cannot do anything about it. Mesa did not react to my bugreport yet.

[devurandom]

In his comment on upstream bug #91130 Serge Martin mentions that a patch series exists to fix the issue upstream.

[magnumripper]

If I understand him right, we'd be OK with that even if they are stubs. As long as we can link a working JtR binary, we can do the right things at run-time.

[magnumripper]

But like I said, we're already fine now (or should be) with the extra check in autoconf. Mesa should fix it to avoid problems with other programs.

[magnumripper]

After 37c88d9 there should be an LM-opencl format testable with Mesa.