kriszyp / cbor-x

Ultra-fast CBOR encoder/decoder with extensions for records and structural cloning
MIT License
264 stars 33 forks source link

Streaming Uint8Array very slow #108

Closed MeirionHughes closed 4 days ago

MeirionHughes commented 3 months ago

Stumbled upon a weird issue when testing through-put, whereby streaming raw Uint8Array into the encoder/decoder causes orders of magnitude performance issue. I can't see anything on the options to force objectMode (which I assume is the issue).

for sake of repo:

import  { EncoderStream, DecoderStream }  from'cbor-x';

let i = 0, j =0, N = 2, M = 1024;

let payloads = new Array(N).fill([]).map(x=>new Uint8Array(M).fill(0).map(y=>Math.random()*255));

const encoderStream = new EncoderStream();
const decoderStream = new DecoderStream();

encoderStream.pipe(decoderStream);

for(let payload of payloads){
  encoderStream.write(payload);
}
encoderStream.end();

let start = Date.now();

decoderStream.on("data", (chunk)=>{
  console.log("read payload", chunk);
});

let clock = setTimeout(()=>{console.log("timed-out")}, 20000);

decoderStream.on("end", ()=>{
  console.log((N*M) + " bytes took ", + Date.now()-start  + " ms");
  clearTimeout(clock);
});

with encoderStream.write(payload); its about 10 seconds, while as object its 4ms encoderStream.write({payload});

This seems to be an issue if you send Float64Array (or any TypedArray) as the message.

Have I missed an option, not supported or bug?

kriszyp commented 1 month ago

Should be addressed in v1.6.