Does not compile on machines without sse4.2

[danez]

The latest version does not compile on our virtual machines anymore. It works with 2.3 but not with 2.4.

error /home/user/project/node_modules/metrohash: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments: 
Directory: /home/user/project/node_modules/metrohash
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@8.11.2 | linux | x64
gyp info spawn /usr/bin/python

gyp info spawn args [ '/home/user/project/node_modules/node-gyp/gyp/gyp_main.py',

gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/user/project/node_modules/metrohash/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/user/project/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/user/.node-gyp/8.11.2/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',

gyp info spawn args   '-Dnode_root_dir=/home/user/.node-gyp/8.11.2',
gyp info spawn args   '-Dnode_gyp_dir=/home/user/project/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/user/.node-gyp/8.11.2/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/user/project/node_modules/metrohash',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory `/home/user/project/node_modules/metrohash/build'
  CXX(target) Release/obj.target/metrohash/src/metrohash_wrapper.o
  CXX(target) Release/obj.target/metrohash/src/metrohash128.o

  CXX(target) Release/obj.target/metrohash/src/metrohash128crc.o

In file included from ../src/metrohash128crc.cpp:27:0:
/usr/lib/gcc/x86_64-linux-gnu/4.8/include/nmmintrin.h:31:3: error: #error "SSE4.2 instruction set not enabled"
 # error "SSE4.2 instruction set not enabled"
   ^
../src/metrohash128crc.cpp: In function ‘void metrohash128crc_1(const uint8_t*, uint64_t, uint32_t, uint8_t*)’:
../src/metrohash128crc.cpp:55:54: error: ‘_mm_crc32_u64’ was not declared in this scope
             v[0] ^= _mm_crc32_u64(v[0], read_u64(ptr)); ptr += 8;
                                                      ^
../src/metrohash128crc.cpp:84:50: error: ‘_mm_crc32_u64’ was not declared in this scope
         v[1] ^= _mm_crc32_u64(v[0], read_u32(ptr)); ptr += 4;
                                                  ^
../src/metrohash128crc.cpp:90:50: error: ‘_mm_crc32_u64’ was not declared in this scope
         v[0] ^= _mm_crc32_u64(v[1], read_u16(ptr)); ptr += 2;
                                                  ^
../src/metrohash128crc.cpp:96:50: error: ‘_mm_crc32_u64’ was not declared in this scope
         v[1] ^= _mm_crc32_u64(v[0], read_u8 (ptr));

                                                  ^
../src/metrohash128crc.cpp: In function ‘void metrohash128crc_2(const uint8_t*, uint64_t, uint32_t, uint8_t*)’:
../src/metrohash128crc.cpp:131:54: error: ‘_mm_crc32_u64’ was not declared in this scope
             v[0] ^= _mm_crc32_u64(v[0], read_u64(ptr)); ptr += 8;
                                                      ^
../src/metrohash128crc.cpp:160:50: error: ‘_mm_crc32_u64’ was not declared in this scope
         v[1] ^= _mm_crc32_u64(v[0], read_u32(ptr)); ptr += 4;
                                                  ^
../src/metrohash128crc.cpp:166:50: error: ‘_mm_crc32_u64’ was not declared in this scope
         v[0] ^= _mm_crc32_u64(v[1], read_u16(ptr)); ptr += 2;
                                                  ^
../src/metrohash128crc.cpp:172:50: error: ‘_mm_crc32_u64’ was not declared in this scope
         v[1] ^= _mm_crc32_u64(v[0], read_u8 (ptr));
                                                  ^
make: *** [Release/obj.target/metrohash/src/metrohash128crc.o] Error 1
make: Leaving directory `/home/user/project/node_modules/metrohash/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/home/user/project/node_modules/node-gyp/lib/build.js:258:23)

gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)

gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Linux 4.4.0-96-generic
gyp ERR! command "/home/user/project/vendor/composer/nodejs/bin/node" "/home/user/project/node_modules/metrohash/node_modules/.bin/node-gyp" "rebuild"
gyp ERR! cwd /home/user/project/node_modules/metrohash
gyp ERR! node -v v8.11.2
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
Error installing dependencies

cat /proc/cpuinfo:

processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 6
model name  : QEMU Virtual CPU version 2.0.0
stepping    : 3
microcode   : 0x1
cpu MHz     : 2499.998
cache size  : 4096 KB
physical id : 0
siblings    : 1
core id     : 0
cpu cores   : 1
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 4
wp      : yes
flags       : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni vmx cx16 x2apic popcnt hypervisor lahf_lm abm tpr_shadow vnmi flexpriority ept vpid
bugs        :
bogomips    : 4999.99
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor   : 1
vendor_id   : GenuineIntel
cpu family  : 6
model       : 6
model name  : QEMU Virtual CPU version 2.0.0
stepping    : 3
microcode   : 0x1
cpu MHz     : 2499.998
cache size  : 4096 KB
physical id : 1
siblings    : 1
core id     : 0
cpu cores   : 1
apicid      : 1
initial apicid  : 1
fpu     : yes
fpu_exception   : yes
cpuid level : 4
wp      : yes
flags       : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni vmx cx16 x2apic popcnt hypervisor lahf_lm abm tpr_shadow vnmi flexpriority ept vpid
bugs        :
bogomips    : 4999.99
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

[robertklep]

This is ultimately an issue with the underlying implementation, for which the original developer states that "[t]he hash function generation software made no effort toward portability".

So sadly, this won't ever compile on non-SSE environments until someone patches the underlying code.

[danez]

But why did it compile with 2.3 of this module?

[robertklep]

Uhm sorry about that, I missed that it worked with 2.3.0.

What might be causing the problem is this change:

-    "cflags"         : [ "-std=c++11 -Wno-deprecated-declarations -Wno-unused-value -Wno-unused-function -Wno-unknown-pragmas -Wno-format -msse4.2" ],
+    "cflags"         : [ "-std=c++11 -Wno-deprecated-declarations -Wno-unused-value -Wno-unused-function -Wno-unknown-pragmas -Wno-format -march=native" ],

Could you check and see if reverting that change (in binding.gyp) fixes your problem?

[danez]

Yes the compiling works. I guess that by reverting this it gets compiled with sse 4.2 support even if the platform does not support it? Which seems also not good.

In our case we haven't really tried running it on this machines without sse 4.2 as we only use metrohash on development machines (but we install it on the servers). If I have to guess I would think that the compiled version won't work there and segfaults or crashes, but cannot test that right now.

[robertklep]

Yeah, I just tried on hardware that doesn't support SSE4.2:

Illegal instruction (core dumped)

[danez]

Ok then let's close this and accept that sse4.2 is a hard requirement. And failing on compile is probably better than failing later during runtime.

Thanks for your time.