Async interface is very slow.

[jodevsa]

Hello again @royaltm ,

Using the asnyc interface is 10 times slower than using a non async function.

i wrote this simple test to illustrate what i mean:

// 'normal' or 'async'
const TEST_TYPE='async';
//const TEST_TYPE='normal';

const {murmurHash32Async, murmurHash64Async, murmurHash128Async} = require('murmurhash-native/promisify')();
const {murmurHash32, murmurHash64, murmurHash128} = require('murmurhash-native');
let total=0;
let current=new Date();
let count=()=>{
  total++;
  const nd=new Date();
if((nd- current)>=1000){
  console.log("done",total,"per second");

  total=0;
  current=nd;
}

};

const asyncTest= async () => {
  for(let i=0;i<100000000000;i++){
    const str="hey" + String(i);
    x=await murmurHash32Async(str);
    count();
  }

};
const normalTest= async () => {
  for(let i=0;i<100000000000;i++){
    const str="hey" + String(i);
    x=await murmurHash32(str);
    count();
  }

};

TEST_TYPE==='async'?asyncTest():normalTest();

[royaltm]

Yes, async interface has an overhead - creating async work queue and a promise object. You should use async interface only when it benefits you: e.g. when running a lot of concurrent jobs and the size of your hashed data is significant. In your test you are hashing very small strings. You need to measure yourself when asynchronous API will start benefiting your use case. See bench.async.js for demonstration:

$ node bench/bench.async.js
parallel threads: 8
test duration: 1000 ms
murmurHash          (string[80]): single: 6.8432MB/s avg: 54.4570MB/s
murmurHash64x86     (string[80]): single: 6.0143MB/s avg: 47.8144MB/s
murmurHash64x64     (string[80]): single: 5.9294MB/s avg: 47.1371MB/s
murmurHash128x86    (string[80]): single: 5.6521MB/s avg: 44.9891MB/s
murmurHash128x64    (string[80]): single: 5.5743MB/s avg: 44.3940MB/s
MurmurHash          (string[80]): single: 4.2346MB/s avg: 33.7440MB/s
MurmurHash128x86    (string[80]): single: 3.6171MB/s avg: 28.8144MB/s
MurmurHash128x64    (string[80]): single: 3.3915MB/s avg: 27.0082MB/s
murmurHash          (string[131072]): single: 1518.1436MB/s avg: 12122.5109MB/s
murmurHash64x86     (string[131072]): single: 1882.0782MB/s avg: 15032.4011MB/s
murmurHash64x64     (string[131072]): single: 2143.0289MB/s avg: 17104.0076MB/s
murmurHash128x86    (string[131072]): single: 1869.0995MB/s avg: 14924.4764MB/s
murmurHash128x64    (string[131072]): single: 2126.7677MB/s avg: 16972.0491MB/s
MurmurHash          (string[131072]): single: 1500.4002MB/s avg: 11976.3290MB/s
MurmurHash128x86    (string[131072]): single: 1656.3788MB/s avg: 13235.5498MB/s
MurmurHash128x64    (string[131072]): single: 1755.1773MB/s avg: 14020.7087MB/s
murmurHash          (buffer[131072]): single: 1583.5955MB/s avg: 12658.3553MB/s
murmurHash64x86     (buffer[131072]): single: 2093.2752MB/s avg: 16708.0321MB/s
murmurHash64x64     (buffer[131072]): single: 3560.2544MB/s avg: 28370.8861MB/s
murmurHash128x86    (buffer[131072]): single: 2071.9017MB/s avg: 16540.5116MB/s
murmurHash128x64    (buffer[131072]): single: 3648.9315MB/s avg: 29101.3552MB/s
MurmurHash          (buffer[131072]): single: 1543.9277MB/s avg: 12334.4894MB/s
MurmurHash128x86    (buffer[131072]): single: 1927.0191MB/s avg: 15370.8558MB/s
MurmurHash128x64    (buffer[131072]): single: 3353.7095MB/s avg: 26767.3880MB/s

single indicates how fast single thread processes its data and avg how all concurrent (in my case 8) threads perform asynchronously combining their throughput. In comparison see how fast perform synchronous (blocking) counterparts in a single thread:

$ node bench/bench.js -n
test duration: 1000 ms
murmurHash              (string[80]): 293.0191MB/s
murmurHash64x86         (string[80]): 211.3339MB/s
murmurHash64x64         (string[80]): 219.3842MB/s
murmurHash128x86        (string[80]): 198.0301MB/s
murmurHash128x64        (string[80]): 203.8096MB/s
MurmurHash              (string[80]): 76.2226MB/s
MurmurHash128x86        (string[80]): 63.5513MB/s
MurmurHash128x64        (string[80]): 63.7050MB/s
murmurHash              (string[131072]): 3150.3432MB/s
murmurHash64x86         (string[131072]): 4447.5529MB/s
murmurHash64x64         (string[131072]): 6344.6041MB/s
murmurHash128x86        (string[131072]): 4225.2752MB/s
murmurHash128x64        (string[131072]): 6761.4893MB/s
MurmurHash              (string[131072]): 3049.3822MB/s
MurmurHash128x86        (string[131072]): 4004.1135MB/s
MurmurHash128x64        (string[131072]): 6211.4664MB/s
murmurHash              (buffer[131072]): 3444.5244MB/s
murmurHash64x86         (buffer[131072]): 5188.7245MB/s
murmurHash64x64         (buffer[131072]): 7737.0529MB/s
murmurHash128x86        (buffer[131072]): 4787.0290MB/s
murmurHash128x64        (buffer[131072]): 8469.0781MB/s
MurmurHash              (buffer[131072]): 3402.9634MB/s
MurmurHash128x86        (buffer[131072]): 4599.0852MB/s
MurmurHash128x64        (buffer[131072]): 7903.8072MB/s

E.g. on my machine when running MurmurHash128x64 on a 131072 bytes length buffer you get synchronous (blocking): 7903 MB/s. At the same time when running asynchronous hashing on the same buffer in a single thread you get only 3353 MB/s. But when you run the asynchronous tasks in parallel you will achieve combined throughput up to 26767 MB/s. For small string (80 ascii characters) you get synchronous throughput: 293 MB/s asynchronous-single: 6.84 MB/s asynchronous-combined only: 54.45MB/s. So to really benefit from asynchronous API you need to have a lot of concurrent tasks running at the same time and the size of the hashed data must be significant (on my computer > 60kb).

[jodevsa]

Thanks alot man, i fully understood when to use the async interface :+1: