search

openwall / john

John the Ripper jumbo - advanced offline password cracker, which supports hundreds of hash and cipher types, and runs on many operating systems, CPUs, GPUs, and even some FPGAs
https://www.openwall.com/john/
Other
10.21k stars 2.09k forks source link

32bit build with 64bit avx2 CPU fails #1205

Closed frank-dittrich closed 9 years ago

frank-dittrich commented 9 years ago

This is a 32bit Linux system (Fedora 20), with an Intel(R) Core(TM) i5-4570 CPU. Autoconf build now fails with

In file included from rawSHA1_ng_fmt_plug.c:25:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
In file included from rawSHA1_ng_fmt_plug.c:48:0:
int128.h:28:2: error: #error No int128 type found supported by the compiler.
 #error No int128 type found supported by the compiler.
  ^
make[1]: *** [rawSHA1_ng_fmt_plug.o] Error 1
make: *** [default] Error 2
frank-dittrich commented 9 years ago

With make -s -j 8, I get more of these errors:

In file included from rawSHA256_ng_fmt_plug.c:10:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
In file included from params.h:24:0,
                 from formats.h:15,
                 from rawSHA512_common_plug.c:11:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
In file included from rawSHA512_fmt_plug.c:23:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
In file included from rawSHA256_fmt_plug.c:25:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
In file included from rawSHA1_ng_fmt_plug.c:25:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
In file included from rawSHA512_ng_fmt_plug.c:10:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
In file included from rawSHA1_ng_fmt_plug.c:48:0:
int128.h:28:2: error: #error No int128 type found supported by the compiler.
 #error No int128 type found supported by the compiler.
  ^
In file included from rawSHA384_fmt_plug.c:23:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
In file included from rawmd5u_fmt_plug.c:18:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^
make[1]: *** [rawSHA1_ng_fmt_plug.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [default] Error 2
frank-dittrich commented 9 years ago

Also, legacy generic build on the same system (which I just fixed a few commits ago) is now broken as well:

$ make -s -f Makefile.legacy generic
Compiling: DES benchmark (code version #1)
In file included from sse-intrinsics.h:18:0,
                 from MD5_fmt.c:17:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from MD5_fmt.c:17:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
Benchmarking: descrypt, traditional crypt(3) [DES 24/32 4K]... 135857 c/s real, 137216 c/s virtual
Compiling: DES benchmark (code version #2)
Benchmarking: descrypt, traditional crypt(3) [DES 24/32 128K]... 315904 c/s real, 315904 c/s virtual
Compiling: DES benchmark (code version #3)
Benchmarking: descrypt, traditional crypt(3) [DES 24/32 4K]... 171722 c/s real, 173440 c/s virtual
Compiling: DES benchmark (code version #4)
Benchmarking: descrypt, traditional crypt(3) [DES 24/32 4K]... 163968 c/s real, 163968 c/s virtual
Compiling: DES benchmark (code version #2, no key copying)
Benchmarking: descrypt, traditional crypt(3) [DES 24/32 128K]... 316160 c/s real, 316160 c/s virtual
Compiling: DES benchmark (bitslice, code version #1)
Benchmarking: descrypt, traditional crypt(3) [DES 32/32]... 676224 c/s real, 676224 c/s virtual
Compiling: DES benchmark (bitslice, code version #2)
Benchmarking: descrypt, traditional crypt(3) [DES 32/32]... 696672 c/s real, 696672 c/s virtual
Compiling: DES benchmark (bitslice, code version #3)
Benchmarking: descrypt, traditional crypt(3) [DES 32/32]... 415104 c/s real, 415104 c/s virtual
Benchmarking: md5crypt, crypt(3) $1$ [MD5 32/32]... 10869 c/s real, 10869 c/s virtual
Compiling: MD5 benchmark (two hashes at a time)
In file included from sse-intrinsics.h:18:0,
                 from MD5_fmt.c:17:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from MD5_fmt.c:17:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
Benchmarking: md5crypt, crypt(3) $1$ [MD5 32/32 X2]... 14208 c/s real, 14208 c/s virtual
Compiling: MD5 benchmark (immediate values)
In file included from sse-intrinsics.h:18:0,
                 from MD5_fmt.c:17:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from MD5_fmt.c:17:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
Benchmarking: md5crypt, crypt(3) $1$ [MD5 32/32 X2]... 13746 c/s real, 13746 c/s virtual
Benchmarking: bcrypt ("$2a$05", 32 iterations) [Blowfish 32/32]... 503 c/s real, 503 c/s virtual
Compiling: Blowfish benchmark (scale)
Benchmarking: bcrypt ("$2a$05", 32 iterations) [Blowfish 32/32]... 554 c/s real, 560 c/s virtual
Compiling: Blowfish benchmark (two hashes at a time)
Benchmarking: bcrypt ("$2a$05", 32 iterations) [Blowfish 32/32 X2]... 896 c/s real, 896 c/s virtual
In file included from sse-intrinsics.h:18:0,
                 from MD5_fmt.c:17:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from MD5_fmt.c:17:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from dynamic.h:30,
                 from dynamic_fmt.c:113:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from dynamic_fmt.c:108:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from dynamic.h:30,
                 from dynamic_parser.c:79:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from dynamic_parser.c:71:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from dynamic.h:30,
                 from dynamic_preloads.c:34:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from dynamic_preloads.c:31:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from dynamic.h:30,
                 from dynamic_utils.c:35:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from dynamic_utils.c:31:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from dynamic.h:30,
                 from dynamic_big_crypt.c:44:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from dynamic_big_crypt.c:28:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from bitcoin_fmt_plug.c:39:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from bitcoin_fmt_plug.c:32:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from ntlmv1_mschapv2_fmt_plug.c:83:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from sse-intrinsics.h:18:0,
                 from pbkdf2_hmac_sha256.h:25,
                 from rar5_fmt_plug.c:42:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from johnswap.h:5,
                 from rar5_fmt_plug.c:32:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from ntlmv1_mschapv2_fmt_plug.c:83:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from hmacSHA512_fmt_plug.c:24:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from hmacSHA512_fmt_plug.c:21:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from pbkdf2_hmac_sha256.h:25,
                 from lastpass_fmt_plug.c:28:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from johnswap.h:5,
                 from lastpass_fmt_plug.c:21:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from pbkdf2_hmac_sha1.h:30,
                 from keychain_fmt_plug.c:28:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from sse-intrinsics.h:18:0,
                 from rawSHA1_fmt_plug.c:37:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from keychain_fmt_plug.c:23:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from common.h:19:0,
                 from rawSHA1_fmt_plug.c:21:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from XSHA512_fmt_plug.c:19:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from XSHA512_fmt_plug.c:16:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from dynamic.h:30,
                 from net_sha1_fmt_plug.c:35:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from dynamic.h:30,
                 from net_sha1_fmt_plug.c:35:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from wpapsk_fmt_plug.c:21:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from wpapsk_fmt_plug.c:21:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from FGT_fmt_plug.c:37:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from FGT_fmt_plug.c:31:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from hmacSHA224_fmt_plug.c:24:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from hmacSHA224_fmt_plug.c:21:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from mscash2_fmt_plug.c:71:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from mscash2_fmt_plug.c:63:0:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from dynamic.h:30,
                 from net_md5_fmt_plug.c:35:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from dynamic.h:30,
                 from net_md5_fmt_plug.c:35:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from dynamic.h:30,
                 from pixMD5_fmt_plug.c:23:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from pixMD5_fmt_plug.c:21:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from nsldap_fmt_plug.c:22:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from nsldap_fmt_plug.c:22:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from mssql12_fmt_plug.c:48:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from mssql12_fmt_plug.c:42:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from SybaseASE_fmt_plug.c:51:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from SybaseASE_fmt_plug.c:46:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
SybaseASE_fmt_plug.c: In function ‘init’:
pseudo_intrinsics.h:445:34: error: ‘SIMD_COEF_32’ undeclared (first use in this function)
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
                                  ^
memory.h:98:53: note: in definition of macro ‘mem_calloc_tiny’
 #define mem_calloc_tiny(a,b) mem_calloc_tiny_func(a,b)
                                                     ^
SybaseASE_fmt_plug.c:115:36: note: in expansion of macro ‘MEM_ALIGN_SIMD’
   self->params.max_keys_per_crypt, MEM_ALIGN_SIMD);
                                    ^
pseudo_intrinsics.h:445:34: note: each undeclared identifier is reported only once for each function it appears in
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
                                  ^
memory.h:98:53: note: in definition of macro ‘mem_calloc_tiny’
 #define mem_calloc_tiny(a,b) mem_calloc_tiny_func(a,b)
                                                     ^
SybaseASE_fmt_plug.c:115:36: note: in expansion of macro ‘MEM_ALIGN_SIMD’
   self->params.max_keys_per_crypt, MEM_ALIGN_SIMD);
                                    ^
make[1]: *** [SybaseASE_fmt_plug.o] Error 1
make[1]: *** Waiting for unfinished jobs....
In file included from sse-intrinsics.h:18:0,
                 from pbkdf2_hmac_sha512.h:22,
                 from cloudkeychain_fmt_plug.c:35:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from cloudkeychain_fmt_plug.c:28:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from pbkdf2_hmac_sha1.h:30,
                 from encfs_fmt_plug.c:26:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from pbkdf2_hmac_sha1.h:30,
                 from encfs_fmt_plug.c:26:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from mysqlSHA1_fmt_plug.c:42:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from mysqlSHA1_fmt_plug.c:42:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from pbkdf2_hmac_sha1.h:30,
                 from zip_fmt_plug.c:59:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from zip_fmt_plug.c:54:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from pwsafe_fmt_plug.c:36:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from pwsafe_fmt_plug.c:31:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
In file included from sse-intrinsics.h:18:0,
                 from rawSHA512_fmt_plug.c:39:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from rawSHA512_fmt_plug.c:27:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
make: *** [generic] Error 2
frank-dittrich commented 9 years ago

And of course, linux-x86-native is broken as well on this system:

In file included from wpapsk_fmt_plug.c:20:0:
arch.h:263:0: warning: "MD4_N_STR" redefined [enabled by default]
 #define MD4_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD4_SSE_PARA)
 ^
arch.h:229:0: note: this is the location of the previous definition
 #define MD4_N_STR   "12x"
 ^
arch.h:268:0: warning: "MD5_N_STR" redefined [enabled by default]
 #define MD5_N_STR   PARA_TO_MxN(SIMD_COEF_32, MD5_SSE_PARA)
 ^
arch.h:204:0: note: this is the location of the previous definition
 #define MD5_N_STR   "12x"
 ^
arch.h:273:0: warning: "SHA1_N_STR" redefined [enabled by default]
 #define SHA1_N_STR   PARA_TO_MxN(SIMD_COEF_32, SHA1_SSE_PARA)
 ^
arch.h:248:0: note: this is the location of the previous definition
 #define SHA1_N_STR   "8x"
 ^

@magnumripper I'll see what needs to be fixed for legacy (generic/native) builds once you looked into the autoconf build.

frank-dittrich commented 9 years ago

BTW: a legacy generic build on a 64bit Linux system fails as well:

$ make -s -f Makefile.legacy generic
Compiling: DES benchmark (code version #1)
In file included from pseudo_intrinsics.h:30:0,
                 from sse-intrinsics.h:18,
                 from MD5_fmt.c:17:
pseudo_intrinsics.h: In function ‘vtestz_epi32’:
pseudo_intrinsics.h:346:21: error: ‘SIMD_COEF_32’ undeclared (first use in this function)
  uint32_t JTR_ALIGN(SIMD_COEF_32 * 4) words[4];
                     ^
aligned.h:16:46: note: in definition of macro ‘JTR_ALIGN’
 #define JTR_ALIGN(n) __attribute__ ((aligned(n)))
                                              ^
pseudo_intrinsics.h:346:21: note: each undeclared identifier is reported only once for each function it appears in
  uint32_t JTR_ALIGN(SIMD_COEF_32 * 4) words[4];
                     ^
aligned.h:16:46: note: in definition of macro ‘JTR_ALIGN’
 #define JTR_ALIGN(n) __attribute__ ((aligned(n)))
                                              ^
In file included from sse-intrinsics.h:18:0,
                 from MD5_fmt.c:17:
pseudo_intrinsics.h:346:2: error: requested alignment is not an integer constant
  uint32_t JTR_ALIGN(SIMD_COEF_32 * 4) words[4];
  ^
pseudo_intrinsics.h: At top level:
pseudo_intrinsics.h:445:0: warning: "MEM_ALIGN_SIMD" redefined [enabled by default]
 #define MEM_ALIGN_SIMD          (SIMD_COEF_32 * 4)
 ^
In file included from common.h:19:0,
                 from sse-intrinsics.h:17,
                 from MD5_fmt.c:17:
memory.h:43:0: note: this is the location of the previous definition
 #define MEM_ALIGN_SIMD   (16)
 ^
make[1]: *** [MD5_fmt.o] Error 1
make: *** [generic.h] Error 1

It succeded before (with commit 8c58013e7ebdbe0b539108b659ac503c8d6b01e2. Just Raw-SHA256-ng format (which wasn't built in a 32bit legacy generic build) failed self test:

Testing: Raw-SHA256-ng [SHA256 128/128 SSE2 4x]... FAILED (cmp_all(1))
magnumripper commented 9 years ago

Try 93428a9

magnumripper commented 9 years ago

4100174 addresses generic build but I can't test with the current logic for mem_alloc_align()

magnumripper commented 9 years ago

I will fix generic build. Very small changes are needed - the trick is to make them at the right places.

magnumripper commented 9 years ago

Who the heck put the -j in here?

MAKE_ORIG = +$(MAKE) -f Makefile.legacy -j$(J)

Can't find it with git blame. It ends up spawning an unlimited number of threads when builidng.

magnumripper commented 9 years ago

I can build generic after 79ab668

magnumripper commented 9 years ago

Who the heck put the -j in here?

Apparently you did, in 7c748e2b.

It ends up spawning an unlimited number of threads when builidng.

Is this working fine on other platforms? What is $(J) supposed to mean? I didn't say -j at all, yet it spawned processes without a limit.

frank-dittrich commented 9 years ago

On 04/15/2015 02:11 AM, magnum wrote:

Who the heck put the -j in here?

|MAKE_ORIG = +$(MAKE) -f Makefile.legacy -j$(J)

Might have been my fault, but must have been a long time ago (way before autoconf was intorduced).

Feel free to drop it.

frank-dittrich commented 9 years ago

It was a long time ago. IIRC, I wanted to fix something when make -j N was used.

magnumripper commented 9 years ago

Feel free to drop it.

That will probably mean it always runs like -j1 (even if -j4 was requested). I have a feeling it currently works fine when eg. -j4 or -j1 is given and the problem is when no -j was given in the first place. So we need to drop it or fix it, but fixing it would be better.

frank-dittrich commented 9 years ago

I testes legacy generic for 32bit and 64bit on Linux. Both work, even including dynamic. Thanks!

frank-dittrich commented 9 years ago

Can you somehow define J to 1 if no -j N was given on the command line? I think I tried that in the past but failed.

frank-dittrich commented 9 years ago

32bit autoconf build with avx2 capable CPU also builds fine now (of course, without dynamic). All formats pass self test.

magnumripper commented 9 years ago

Good! It's a weird build - who would want to run a 32-bit OS on a Haswell? But anyway it's great it's working, I expected more problems.

Now if only Solar could commit his DES patches for AVX2...

frank-dittrich commented 9 years ago

Weird reason indeed. I had an old atom CPU netbook which I used for 32bit tests for a long time. Finally, this netbook stopped working. Just for fun, I inserted the HD into my PC with the i5 CPU, booted from it (something that probably wouldn't work with a Windos installation), ran a yum update, rebooted, pulled the latest git changes and tested 32bit builds.

magnumripper commented 9 years ago

So can we close this?

frank-dittrich commented 9 years ago

I think we have a state where all that worked prior to merging the avx2 changes works again. A few legacy build targets still can't be built or result in failing self tests for a few formats, but that is a separate "problem", and was the case even before you merged the avx2 changes. If I have time to dig into the remaining problems, I'll do it without the need of an open issue.