cryptocoinjs / keccak

Keccak sponge function family
MIT License
86 stars 24 forks source link

Benchmark shows that pure JS implementation is faster than native bindings. #4

Closed cakoose closed 6 years ago

cakoose commented 6 years ago

I expected the native bindings would be faster than the pure JS implementation but when I run the benchmark, it shows the opposite. Perhaps I'm not building the native bindings correctly?

Benchmark results:

Bindings (current) x 161,361 ops/sec ±1.12% (87 runs sampled)
Pure JS (current) x 181,867 ops/sec ±1.20% (88 runs sampled)
Bindings (sha3) x 192,293 ops/sec ±3.20% (83 runs sampled)
Pure JS (js-sha3) x 261,088 ops/sec ±0.44% (92 runs sampled)
Buffer 0bytes: fastest is Pure JS (js-sha3)
Bindings (current) x 3.71 ops/sec ±0.54% (14 runs sampled)
Pure JS (current) x 4.47 ops/sec ±1.61% (16 runs sampled)
Bindings (sha3) x 3.50 ops/sec ±2.44% (13 runs sampled)
Pure JS (js-sha3) x 4.55 ops/sec ±1.17% (16 runs sampled)
Buffer 10MiB: fastest is Pure JS (js-sha3),Pure JS (current)

My system: Node v8.11.1, macOS 10.13.4.

Steps to reproduce:

$ git clone git@github.com:cryptocoinjs/keccak.git
Cloning into 'keccak'...
remote: Counting objects: 227, done.
remote: Total 227 (delta 0), reused 0 (delta 0), pack-reused 227
Receiving objects: 100% (227/227), 44.04 KiB | 1.33 MiB/s, done.
Resolving deltas: 100% (123/123), done.
$ cd keccak/
$ git log -n1
commit 256cefd8862f731f2385e7883da6e32bb789c4cd
Author: Kirill Fomichev <fanatid@ya.ru>
CDate:  Sat Dec 2 10:56:13 2017

    1.4.0
$ npm install

> keccak@1.4.0 install /Users/user/keccak
> npm run rebuild || echo "Keccak bindings compilation fail. Pure JS implementation will be used."

> keccak@1.4.0 rebuild /Users/user/keccak
> node-gyp rebuild

  CXX(target) Release/obj.target/keccak/src/addon.o
  CC(target) Release/obj.target/keccak/src/libkeccak/KeccakSponge.o
  CC(target) Release/obj.target/keccak/src/libkeccak/KeccakP-1600-reference.o
../src/libkeccak/KeccakP-1600-reference.c:231:13: warning: unused function 'fromBytesToWords' [-Wunused-function]
static void fromBytesToWords(tKeccakLane *stateAsWords, const unsigned char *state)
            ^
../src/libkeccak/KeccakP-1600-reference.c:242:13: warning: unused function 'fromWordsToBytes' [-Wunused-function]
static void fromWordsToBytes(unsigned char *state, const tKeccakLane *stateAsWords)
            ^
2 warnings generated.
  SOLINK_MODULE(target) Release/keccak.node
npm notice created a lockfile as package-lock.json. You should commit this file.
added 651 packages in 13.379s
$ npm install

> sha3@1.2.2 install /Users/user/keccak/benchmarks/node_modules/sha3
> node-gyp rebuild

node   CXX(target) Release/obj.target/sha3/src/addon.o
inde  CXX(target) Release/obj.target/sha3/src/displayIntermediateValues.o
      CXX(target) Release/obj.target/sha3/src/KeccakF-1600-reference.o
  CXX(target) Release/obj.target/sha3/src/KeccakNISTInterface.o
  CXX(target) Release/obj.target/sha3/src/KeccakSponge.o

  SOLINK_MODULE(target) Release/sha3.node
npm notice created a lockfile as package-lock.json. You should commit this file.
added 6 packages in 3.602s
$ node index.js 
Bindings (current) x 161,361 ops/sec ±1.12% (87 runs sampled)
Pure JS (current) x 181,867 ops/sec ±1.20% (88 runs sampled)
Bindings (sha3) x 192,293 ops/sec ±3.20% (83 runs sampled)
Pure JS (js-sha3) x 261,088 ops/sec ±0.44% (92 runs sampled)
Buffer 0bytes: fastest is Pure JS (js-sha3)
Bindings (current) x 3.71 ops/sec ±0.54% (14 runs sampled)
Pure JS (current) x 4.47 ops/sec ±1.61% (16 runs sampled)
Bindings (sha3) x 3.50 ops/sec ±2.44% (13 runs sampled)
Pure JS (js-sha3) x 4.55 ops/sec ±1.17% (16 runs sampled)
Buffer 10MiB: fastest is Pure JS (js-sha3),Pure JS (current)
fanatid commented 6 years ago

Bindings are working, but when you use bindings overhead added in converting data between node objects and C/C++ objects. So, sometimes bindings is significantly slow than pure JS implementation. Performance also depends from different factors, for example, this is my benchmark: node 10.4.0, linux with kernel 4.16.13-300.fc28.x86_64

Bindings (current) x 259,824 ops/sec ±5.67% (77 runs sampled)
Pure JS (current) x 146,439 ops/sec ±7.23% (74 runs sampled)
Bindings (sha3) x 254,129 ops/sec ±6.25% (71 runs sampled)
Pure JS (js-sha3) x 137,107 ops/sec ±56.66% (76 runs sampled)
Buffer 0bytes: fastest is Bindings (current),Bindings (sha3)
Bindings (current) x 7.72 ops/sec ±3.26% (23 runs sampled)
Pure JS (current) x 3.63 ops/sec ±3.71% (14 runs sampled)
Bindings (sha3) x 7.55 ops/sec ±4.52% (23 runs sampled)
Pure JS (js-sha3) x 4.28 ops/sec ±1.74% (15 runs sampled)
Buffer 10MiB: fastest is Bindings (current),Bindings (sha3)