search

nodejs / node

Node.js JavaScript runtime ✨🐢🚀✨
https://nodejs.org
Other
106.93k stars 29.17k forks source link

TextDecoder is slow. #39879

Open jimmywarting opened 3 years ago

jimmywarting commented 3 years ago

in #39301 we talked about marking string_decoder as legacy but we also brougth up the perf of String_decoder vs TextDecoder, but we didn't talk much about if there was something wrong with TextDecoder and how it performed agains other environments and why it's slower. or if it can be approved upon.

I conducted a test that runs 10k times in different environment and tested to see how fast it is able to do it's job and nodejs didn't do a good job...

here is the results i got

smallUint8 bigUint8
chrome 78 ms 1259 ms
safari 70 ms 441 ms
Firefox 92 ms 2889 ms
NodeJS 360 ms 4167 ms
Deno 569 ms 1386 ms

(Lower is better) buffer.toString('utf8') and string_decoder is considerable much faster, maybe you can steel something from it and apply it onto the TextDecoder? or do exactly what safari is doing :)

And the test i run: ```js const {bigUint8, smallUint8} = fixture() const decoder = new TextDecoder() console.time('small TextDe decode') for (let i = 0; i < 100000; i++) decoder.decode(smallUint8) console.timeEnd('small TextDe decode') console.time('big TextDe decode') for (let i = 0; i < 100000; i++) decoder.decode(bigUint8) console.timeEnd('big TextDe decode') function fixture () { const smallUint8 = new Uint8Array([226, 132, 149, 32, 226, 138, 134, 32, 226, 132, 149, 226, 130, 128, 32, 226, 138, 130, 32, 226, 132, 164, 32, 226, 138, 130, 32, 226, 132, 154, 32, 226, 138, 130, 32, 226, 132, 157, 32, 226, 138, 130, 32, 226, 132, 130, 44, 32, 226, 138, 165, 32, 60, 32, 97, 32, 226, 137, 160, 32, 98, 32, 226, 137, 161, 32, 99, 32, 226, 137, 164, 32, 100, 32, 226, 137, 170, 32, 226, 138, 164, 32, 226, 135, 146, 32, 40, 65, 32, 226, 135, 148, 32, 66, 41, 44]) const bigUint8 = new Uint8Array([60,33,68,79,67,84,89,80,69,32,104,116,109,108,62,10,60,104,116,109,108,32,108,97,110,103,61,34,101,110,34,62,10,10,60,104,101,97,100,62,10,32,32,32,32,60,109,101,116,97,32,99,104,97,114,115,101,116,61,34,85,84,70,45,56,34,62,10,32,32,32,32,60,116,105,116,108,101,62,104,116,116,112,98,105,110,46,111,114,103,60,47,116,105,116,108,101,62,10,32,32,32,32,60,108,105,110,107,32,104,114,101,102,61,34,104,116,116,112,115,58,47,47,102,111,110,116,115,46,103,111,111,103,108,101,97,112,105,115,46,99,111,109,47,99,115,115,63,102,97,109,105,108,121,61,79,112,101,110,43,83,97,110,115,58,52,48,48,44,55,48,48,124,83,111,117,114,99,101,43,67,111,100,101,43,80,114,111,58,51,48,48,44,54,48,48,124,84,105,116,105,108,108,105,117,109,43,87,101,98,58,52,48,48,44,54,48,48,44,55,48,48,34,10,32,32,32,32,32,32,32,32,114,101,108,61,34,115,116,121,108,101,115,104,101,101,116,34,62,10,32,32,32,32,60,108,105,110,107,32,114,101,108,61,34,115,116,121,108,101,115,104,101,101,116,34,32,116,121,112,101,61,34,116,101,120,116,47,99,115,115,34,32,104,114,101,102,61,34,47,102,108,97,115,103,103,101,114,95,115,116,97,116,105,99,47,115,119,97,103,103,101,114,45,117,105,46,99,115,115,34,62,10,32,32,32,32,60,108,105,110,107,32,114,101,108,61,34,105,99,111,110,34,32,116,121,112,101,61,34,105,109,97,103,101,47,112,110,103,34,32,104,114,101,102,61,34,47,115,116,97,116,105,99,47,102,97,118,105,99,111,110,46,105,99,111,34,32,115,105,122,101,115,61,34,54,52,120,54,52,32,51,50,120,51,50,32,49,54,120,49,54,34,32,47,62,10,32,32,32,32,60,115,116,121,108,101,62,10,32,32,32,32,32,32,32,32,104,116,109,108,32,123,10,32,32,32,32,32,32,32,32,32,32,32,32,98,111,120,45,115,105,122,105,110,103,58,32,98,111,114,100,101,114,45,98,111,120,59,10,32,32,32,32,32,32,32,32,32,32,32,32,111,118,101,114,102,108,111,119,58,32,45,109,111,122,45,115,99,114,111,108,108,98,97,114,115,45,118,101,114,116,105,99,97,108,59,10,32,32,32,32,32,32,32,32,32,32,32,32,111,118,101,114,102,108,111,119,45,121,58,32,115,99,114,111,108,108,59,10,32,32,32,32,32,32,32,32,125,10,10,32,32,32,32,32,32,32,32,42,44,10,32,32,32,32,32,32,32,32,42,58,98,101,102,111,114,101,44,10,32,32,32,32,32,32,32,32,42,58,97,102,116,101,114,32,123,10,32,32,32,32,32,32,32,32,32,32,32,32,98,111,120,45,115,105,122,105,110,103,58,32,105,110,104,101,114,105,116,59,10,32,32,32,32,32,32,32,32,125,10,10,32,32,32,32,32,32,32,32,98,111,100,121,32,123,10,32,32,32,32,32,32,32,32,32,32,32,32,109,97,114,103,105,110,58,32,48,59,10,32,32,32,32,32,32,32,32,32,32,32,32,98,97,99,107,103,114,111,117,110,100,58,32,35,102,97,102,97,102,97,59,10,32,32,32,32,32,32,32,32,125,10,32,32,32,32,60,47,115,116,121,108,101,62,10,60,47,104,101,97,100,62,10,10,60,98,111,100,121,62,10,32,32,32,32,60,97,32,104,114,101,102,61,34,104,116,116,112,115,58,47,47,103,105,116,104,117,98,46,99,111,109,47,114,101,113,117,101,115,116,115,47,104,116,116,112,98,105,110,34,32,99,108,97,115,115,61,34,103,105,116,104,117,98,45,99,111,114,110,101,114,34,32,97,114,105,97,45,108,97,98,101,108,61,34,86,105,101,119,32,115,111,117,114,99,101,32,111,110,32,71,105,116,104,117,98,34,62,10,32,32,32,32,32,32,32,32,60,115,118,103,32,119,105,100,116,104,61,34,56,48,34,32,104,101,105,103,104,116,61,34,56,48,34,32,118,105,101,119,66,111,120,61,34,48,32,48,32,50,53,48,32,50,53,48,34,32,115,116,121,108,101,61,34,102,105,108,108,58,35,49,53,49,53,49,51,59,32,99,111,108,111,114,58,35,102,102,102,59,32,112,111,115,105,116,105,111,110,58,32,97,98,115,111,108,117,116,101,59,32,116,111,112,58,32,48,59,32,98,111,114,100,101,114,58,32,48,59,32,114,105,103,104,116,58,32,48,59,34,10,32,32,32,32,32,32,32,32,32,32,32,32,97,114,105,97,45,104,105,100,100,101,110,61,34,116,114,117,101,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,48,44,48,32,76,49,49,53,44,49,49,53,32,76,49,51,48,44,49,49,53,32,76,49,52,50,44,49,52,50,32,76,50,53,48,44,50,53,48,32,76,50,53,48,44,48,32,90,34,62,60,47,112,97,116,104,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,50,56,46,51,44,49,48,57,46,48,32,67,49,49,51,46,56,44,57,57,46,55,32,49,49,57,46,48,44,56,57,46,54,32,49,49,57,46,48,44,56,57,46,54,32,67,49,50,50,46,48,44,56,50,46,55,32,49,50,48,46,53,44,55,56,46,54,32,49,50,48,46,53,44,55,56,46,54,32,67,49,49,57,46,50,44,55,50,46,48,32,49,50,51,46,52,44,55,54,46,51,32,49,50,51,46,52,44,55,54,46,51,32,67,49,50,55,46,51,44,56,48,46,57,32,49,50,53,46,53,44,56,55,46,51,32,49,50,53,46,53,44,56,55,46,51,32,67,49,50,50,46,57,44,57,55,46,54,32,49,51,48,46,54,44,49,48,49,46,57,32,49,51,52,46,52,44,49,48,51,46,50,34,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,102,105,108,108,61,34,99,117,114,114,101,110,116,67,111,108,111,114,34,32,115,116,121,108,101,61,34,116,114,97,110,115,102,111,114,109,45,111,114,105,103,105,110,58,32,49,51,48,112,120,32,49,48,54,112,120,59,34,32,99,108,97,115,115,61,34,111,99,116,111,45,97,114,109,34,62,60,47,112,97,116,104,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,49,53,46,48,44,49,49,53,46,48,32,67,49,49,52,46,57,44,49,49,53,46,49,32,49,49,56,46,55,44,49,49,54,46,53,32,49,49,57,46,56,44,49,49,53,46,52,32,76,49,51,51,46,55,44,49,48,49,46,54,32,67,49,51,54,46,57,44,57,57,46,50,32,49,51,57,46,57,44,57,56,46,52,32,49,52,50,46,50,44,57,56,46,54,32,67,49,51,51,46,56,44,56,56,46,48,32,49,50,55,46,53,44,55,52,46,52,32,49,52,51,46,56,44,53,56,46,48,32,67,49,52,56,46,53,44,53,51,46,52,32,49,53,52,46,48,44,53,49,46,50,32,49,53,57,46,55,44,53,49,46,48,32,67,49,54,48,46,51,44,52,57,46,52,32,49,54,51,46,50,44,52,51,46,54,32,49,55,49,46,52,44,52,48,46,49,32,67,49,55,49,46,52,44,52,48,46,49,32,49,55,54,46,49,44,52,50,46,53,32,49,55,56,46,56,44,53,54,46,50,32,67,49,56,51,46,49,44,53,56,46,54,32,49,56,55,46,50,44,54,49,46,56,32,49,57,48,46,57,44,54,53,46,52,32,67,49,57,52,46,53,44,54,57,46,48,32,49,57,55,46,55,44,55,51,46,50,32,50,48,48,46,49,44,55,55,46,54,32,67,50,49,51,46,56,44,56,48,46,50,32,50,49,54,46,51,44,56,52,46,57,32,50,49,54,46,51,44,56,52,46,57,32,67,50,49,50,46,55,44,57,51,46,49,32,50,48,54,46,57,44,57,54,46,48,32,50,48,53,46,52,44,57,54,46,54,32,67,50,48,53,46,49,44,49,48,50,46,52,32,50,48,51,46,48,44,49,48,55,46,56,32,49,57,56,46,51,44,49,49,50,46,53,32,67,49,56,49,46,57,44,49,50,56,46,57,32,49,54,56,46,51,44,49,50,50,46,53,32,49,53,55,46,55,44,49,49,52,46,49,32,67,49,53,55,46,57,44,49,49,54,46,57,32,49,53,54,46,55,44,49,50,48,46,57,32,49,53,50,46,55,44,49,50,52,46,57,32,76,49,52,49,46,48,44,49,51,54,46,53,32,67,49,51,57,46,56,44,49,51,55,46,55,32,49,52,49,46,54,44,49,52,49,46,57,32,49,52,49,46,56,44,49,52,49,46,56,32,90,34,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,102,105,108,108,61,34,99,117,114,114,101,110,116,67,111,108,111,114,34,32,99,108,97,115,115,61,34,111,99,116,111,45,98,111,100,121,34,62,60,47,112,97,116,104,62,10,32,32,32,32,32,32,32,32,60,47,115,118,103,62,10,32,32,32,32,60,47,97,62,10,32,32,32,32,60,115,118,103,32,120,109,108,110,115,61,34,104,116,116,112,58,47,47,119,119,119,46,119,51,46,111,114,103,47,50,48,48,48,47,115,118,103,34,32,120,109,108,110,115,58,120,108,105,110,107,61,34,104,116,116,112,58,47,47,119,119,119,46,119,51,46,111,114,103,47,49,57,57,57,47,120,108,105,110,107,34,32,115,116,121,108,101,61,34,112,111,115,105,116,105,111,110,58,97,98,115,111,108,117,116,101,59,119,105,100,116,104,58,48,59,104,101,105,103,104,116,58,48,34,62,10,32,32,32,32,32,32,32,32,60,100,101,102,115,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,115,121,109,98,111,108,32,118,105,101,119,66,111,120,61,34,48,32,48,32,50,48,32,50,48,34,32,105,100,61,34,117,110,108,111,99,107,101,100,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,53,46,56,32,56,72,49,52,86,53,46,54,67,49,52,32,50,46,55,48,51,32,49,50,46,54,54,53,32,49,32,49,48,32,49,32,55,46,51,51,52,32,49,32,54,32,50,46,55,48,51,32,54,32,53,46,54,86,54,104,50,118,45,46,56,48,49,67,56,32,51,46,55,53,52,32,56,46,55,57,55,32,51,32,49,48,32,51,99,49,46,50,48,51,32,48,32,50,32,46,55,53,52,32,50,32,50,46,49,57,57,86,56,72,52,99,45,46,53,53,51,32,48,45,49,32,46,54,52,54,45,49,32,49,46,49,57,57,86,49,55,99,48,32,46,53,52,57,46,52,50,56,32,49,46,49,51,57,46,57,53,49,32,49,46,51,48,55,108,49,46,49,57,55,46,51,56,55,67,53,46,54,55,50,32,49,56,46,56,54,49,32,54,46,53,53,32,49,57,32,55,46,49,32,49,57,104,53,46,56,99,46,53,52,57,32,48,32,49,46,52,50,56,45,46,49,51,57,32,49,46,57,53,49,45,46,51,48,55,108,49,46,49,57,54,45,46,51,56,55,99,46,53,50,52,45,46,49,54,55,46,57,53,51,45,46,55,53,55,46,57,53,51,45,49,46,51,48,54,86,57,46,49,57,57,67,49,55,32,56,46,54,52,54,32,49,54,46,51,53,50,32,56,32,49,53,46,56,32,56,122,34,62,60,47,112,97,116,104,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,121,109,98,111,108,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,60,115,121,109,98,111,108,32,118,105,101,119,66,111,120,61,34,48,32,48,32,50,48,32,50,48,34,32,105,100,61,34,108,111,99,107,101,100,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,53,46,56,32,56,72,49,52,86,53,46,54,67,49,52,32,50,46,55,48,51,32,49,50,46,54,54,53,32,49,32,49,48,32,49,32,55,46,51,51,52,32,49,32,54,32,50,46,55,48,51,32,54,32,53,46,54,86,56,72,52,99,45,46,53,53,51,32,48,45,49,32,46,54,52,54,45,49,32,49,46,49,57,57,86,49,55,99,48,32,46,53,52,57,46,52,50,56,32,49,46,49,51,57,46,57,53,49,32,49,46,51,48,55,108,49,46,49,57,55,46,51,56,55,67,53,46,54,55,50,32,49,56,46,56,54,49,32,54,46,53,53,32,49,57,32,55,46,49,32,49,57,104,53,46,56,99,46,53,52,57,32,48,32,49,46,52,50,56,45,46,49,51,57,32,49,46,57,53,49,45,46,51,48,55,108,49,46,49,57,54,45,46,51,56,55,99,46,53,50,52,45,46,49,54,55,46,57,53,51,45,46,55,53,55,46,57,53,51,45,49,46,51,48,54,86,57,46,49,57,57,67,49,55,32,56,46,54,52,54,32,49,54,46,51,53,50,32,56,32,49,53,46,56,32,56,122,77,49,50,32,56,72,56,86,53,46,49,57,57,67,56,32,51,46,55,53,52,32,56,46,55,57,55,32,51,32,49,48,32,51,99,49,46,50,48,51,32,48,32,50,32,46,55,53,52,32,50,32,50,46,49,57,57,86,56,122,34,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,121,109,98,111,108,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,60,115,121,109,98,111,108,32,118,105,101,119,66,111,120,61,34,48,32,48,32,50,48,32,50,48,34,32,105,100,61,34,99,108,111,115,101,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,52,46,51,52,56,32,49,52,46,56,52,57,99,45,46,52,54,57,46,52,54,57,45,49,46,50,50,57,46,52,54,57,45,49,46,54,57,55,32,48,76,49,48,32,49,49,46,56,49,57,108,45,50,46,54,53,49,32,51,46,48,50,57,99,45,46,52,54,57,46,52,54,57,45,49,46,50,50,57,46,52,54,57,45,49,46,54,57,55,32,48,45,46,52,54,57,45,46,52,54,57,45,46,52,54,57,45,49,46,50,50,57,32,48,45,49,46,54,57,55,108,50,46,55,53,56,45,51,46,49,53,45,50,46,55,53,57,45,51,46,49,53,50,99,45,46,52,54,57,45,46,52,54,57,45,46,52,54,57,45,49,46,50,50,56,32,48,45,49,46,54,57,55,46,52,54,57,45,46,52,54,57,32,49,46,50,50,56,45,46,52,54,57,32,49,46,54,57,55,32,48,76,49,48,32,56,46,49,56,51,108,50,46,54,53,49,45,51,46,48,51,49,99,46,52,54,57,45,46,52,54,57,32,49,46,50,50,56,45,46,52,54,57,32,49,46,54,57,55,32,48,32,46,52,54,57,46,52,54,57,46,52,54,57,32,49,46,50,50,57,32,48,32,49,46,54,57,55,108,45,50,46,55,53,56,32,51,46,49,53,50,32,50,46,55,53,56,32,51,46,49,53,99,46,52,54,57,46,52,54,57,46,52,54,57,32,49,46,50,50,57,32,48,32,49,46,54,57,56,122,34,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,121,109,98,111,108,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,60,115,121,109,98,111,108,32,118,105,101,119,66,111,120,61,34,48,32,48,32,50,48,32,50,48,34,32,105,100,61,34,108,97,114,103,101,45,97,114,114,111,119,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,51,46,50,53,32,49,48,76,54,46,49,48,57,32,50,46,53,56,99,45,46,50,54,56,45,46,50,55,45,46,50,54,56,45,46,55,48,55,32,48,45,46,57,55,57,46,50,54,56,45,46,50,55,46,55,48,49,45,46,50,55,46,57,54,57,32,48,108,55,46,56,51,32,55,46,57,48,56,99,46,50,54,56,46,50,55,49,46,50,54,56,46,55,48,57,32,48,32,46,57,55,57,108,45,55,46,56,51,32,55,46,57,48,56,99,45,46,50,54,56,46,50,55,49,45,46,55,48,49,46,50,55,45,46,57,54,57,32,48,45,46,50,54,56,45,46,50,54,57,45,46,50,54,56,45,46,55,48,55,32,48,45,46,57,55,57,76,49,51,46,50,53,32,49,48,122,34,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,121,109,98,111,108,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,60,115,121,109,98,111,108,32,118,105,101,119,66,111,120,61,34,48,32,48,32,50,48,32,50,48,34,32,105,100,61,34,108,97,114,103,101,45,97,114,114,111,119,45,100,111,119,110,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,55,46,52,49,56,32,54,46,49,48,57,99,46,50,55,50,45,46,50,54,56,46,55,48,57,45,46,50,54,56,46,57,55,57,32,48,115,46,50,55,49,46,55,48,49,32,48,32,46,57,54,57,108,45,55,46,57,48,56,32,55,46,56,51,99,45,46,50,55,46,50,54,56,45,46,55,48,55,46,50,54,56,45,46,57,55,57,32,48,108,45,55,46,57,48,56,45,55,46,56,51,99,45,46,50,55,45,46,50,54,56,45,46,50,55,45,46,55,48,49,32,48,45,46,57,54,57,46,50,55,49,45,46,50,54,56,46,55,48,57,45,46,50,54,56,46,57,55,57,32,48,76,49,48,32,49,51,46,50,53,108,55,46,52,49,56,45,55,46,49,52,49,122,34,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,121,109,98,111,108,62,10,10,10,32,32,32,32,32,32,32,32,32,32,32,32,60,115,121,109,98,111,108,32,118,105,101,119,66,111,120,61,34,48,32,48,32,50,52,32,50,52,34,32,105,100,61,34,106,117,109,112,45,116,111,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,57,32,55,118,52,72,53,46,56,51,108,51,46,53,56,45,51,46,53,57,76,56,32,54,108,45,54,32,54,32,54,32,54,32,49,46,52,49,45,49,46,52,49,76,53,46,56,51,32,49,51,72,50,49,86,55,122,34,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,121,109,98,111,108,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,60,115,121,109,98,111,108,32,118,105,101,119,66,111,120,61,34,48,32,48,32,50,52,32,50,52,34,32,105,100,61,34,101,120,112,97,110,100,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,97,116,104,32,100,61,34,77,49,48,32,49,56,104,52,118,45,50,104,45,52,118,50,122,77,51,32,54,118,50,104,49,56,86,54,72,51,122,109,51,32,55,104,49,50,118,45,50,72,54,118,50,122,34,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,121,109,98,111,108,62,10,10,32,32,32,32,32,32,32,32,60,47,100,101,102,115,62,10,32,32,32,32,60,47,115,118,103,62,10,10,10,32,32,32,32,60,100,105,118,32,105,100,61,34,115,119,97,103,103,101,114,45,117,105,34,62,10,32,32,32,32,32,32,32,32,60,100,105,118,32,100,97,116,97,45,114,101,97,99,116,114,111,111,116,61,34,34,32,99,108,97,115,115,61,34,115,119,97,103,103,101,114,45,117,105,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,34,105,110,102,111,114,109,97,116,105,111,110,45,99,111,110,116,97,105,110,101,114,32,119,114,97,112,112,101,114,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,115,101,99,116,105,111,110,32,99,108,97,115,115,61,34,98,108,111,99,107,32,99,111,108,45,49,50,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,34,105,110,102,111,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,104,103,114,111,117,112,32,99,108,97,115,115,61,34,109,97,105,110,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,104,50,32,99,108,97,115,115,61,34,116,105,116,108,101,34,62,104,116,116,112,98,105,110,46,111,114,103,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,115,109,97,108,108,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,114,101,32,99,108,97,115,115,61,34,118,101,114,115,105,111,110,34,62,48,46,57,46,50,60,47,112,114,101,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,109,97,108,108,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,104,50,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,114,101,32,99,108,97,115,115,61,34,98,97,115,101,45,117,114,108,34,62,91,32,66,97,115,101,32,85,82,76,58,32,104,116,116,112,98,105,110,46,111,114,103,47,32,93,60,47,112,114,101,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,104,103,114,111,117,112,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,34,100,101,115,99,114,105,112,116,105,111,110,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,34,109,97,114,107,100,111,119,110,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,112,62,65,32,115,105,109,112,108,101,32,72,84,84,80,32,82,101,113,117,101,115,116,32,38,97,109,112,59,32,82,101,115,112,111,110,115,101,32,83,101,114,118,105,99,101,46,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,98,114,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,98,114,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,98,62,82,117,110,32,108,111,99,97,108,108,121,58,32,60,47,98,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,99,111,100,101,62,36,32,100,111,99,107,101,114,32,114,117,110,32,45,112,32,56,48,58,56,48,32,107,101,110,110,101,116,104,114,101,105,116,122,47,104,116,116,112,98,105,110,60,47,99,111,100,101,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,112,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,97,32,104,114,101,102,61,34,104,116,116,112,115,58,47,47,107,101,110,110,101,116,104,114,101,105,116,122,46,111,114,103,34,32,116,97,114,103,101,116,61,34,95,98,108,97,110,107,34,62,116,104,101,32,100,101,118,101,108,111,112,101,114,32,45,32,87,101,98,115,105,116,101,60,47,97,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,97,32,104,114,101,102,61,34,109,97,105,108,116,111,58,109,101,64,107,101,110,110,101,116,104,114,101,105,116,122,46,111,114,103,34,62,83,101,110,100,32,101,109,97,105,108,32,116,111,32,116,104,101,32,100,101,118,101,108,111,112,101,114,60,47,97,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,33,45,45,32,65,68,68,83,32,84,72,69,32,76,79,65,68,69,82,32,83,80,73,78,78,69,82,32,45,45,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,34,108,111,97,100,105,110,103,45,99,111,110,116,97,105,110,101,114,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,34,108,111,97,100,105,110,103,34,62,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,101,99,116,105,111,110,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,60,47,100,105,118,62,10,10,10,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,39,115,119,97,103,103,101,114,45,117,105,39,62,10,32,32,32,32,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,34,119,114,97,112,112,101,114,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,115,101,99,116,105,111,110,32,99,108,97,115,115,61,34,99,108,101,97,114,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,115,112,97,110,32,115,116,121,108,101,61,34,102,108,111,97,116,58,32,114,105,103,104,116,59,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,91,80,111,119,101,114,101,100,32,98,121,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,97,32,116,97,114,103,101,116,61,34,95,98,108,97,110,107,34,32,104,114,101,102,61,34,104,116,116,112,115,58,47,47,103,105,116,104,117,98,46,99,111,109,47,114,111,99,104,97,99,98,114,117,110,111,47,102,108,97,115,103,103,101,114,34,62,70,108,97,115,103,103,101,114,60,47,97,62,93,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,98,114,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,112,97,110,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,115,101,99,116,105,111,110,62,10,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,60,47,100,105,118,62,10,10,10,10,32,32,32,32,60,115,99,114,105,112,116,32,115,114,99,61,34,47,102,108,97,115,103,103,101,114,95,115,116,97,116,105,99,47,115,119,97,103,103,101,114,45,117,105,45,98,117,110,100,108,101,46,106,115,34,62,32,60,47,115,99,114,105,112,116,62,10,32,32,32,32,60,115,99,114,105,112,116,32,115,114,99,61,34,47,102,108,97,115,103,103,101,114,95,115,116,97,116,105,99,47,115,119,97,103,103,101,114,45,117,105,45,115,116,97,110,100,97,108,111,110,101,45,112,114,101,115,101,116,46,106,115,34,62,32,60,47,115,99,114,105,112,116,62,10,32,32,32,32,60,115,99,114,105,112,116,32,115,114,99,61,39,47,102,108,97,115,103,103,101,114,95,115,116,97,116,105,99,47,108,105,98,47,106,113,117,101,114,121,46,109,105,110,46,106,115,39,32,116,121,112,101,61,39,116,101,120,116,47,106,97,118,97,115,99,114,105,112,116,39,62,60,47,115,99,114,105,112,116,62,10,32,32,32,32,60,115,99,114,105,112,116,62,10,10,32,32,32,32,32,32,32,32,119,105,110,100,111,119,46,111,110,108,111,97,100,32,61,32,102,117,110,99,116,105,111,110,32,40,41,32,123,10,32,32,32,32,32,32,32,32,32,32,32,32,10,10,32,32,32,32,32,32,32,32,32,32,32,32,102,101,116,99,104,40,34,47,115,112,101,99,46,106,115,111,110,34,41,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,46,116,104,101,110,40,102,117,110,99,116,105,111,110,32,40,114,101,115,112,111,110,115,101,41,32,123,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,114,101,115,112,111,110,115,101,46,106,115,111,110,40,41,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,46,116,104,101,110,40,102,117,110,99,116,105,111,110,32,40,106,115,111,110,41,32,123,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,118,97,114,32,99,117,114,114,101,110,116,95,112,114,111,116,111,99,111,108,32,61,32,119,105,110,100,111,119,46,108,111,99,97,116,105,111,110,46,112,114,111,116,111,99,111,108,46,115,108,105,99,101,40,48,44,32,45,49,41,59,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,105,102,32,40,106,115,111,110,46,115,99,104,101,109,101,115,91,48,93,32,33,61,32,99,117,114,114,101,110,116,95,112,114,111,116,111,99,111,108,41,32,123,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,47,47,32,83,119,105,116,99,104,101,115,32,115,99,104,101,109,101,32,116,111,32,116,104,101,32,99,117,114,114,101,110,116,32,105,110,32,117,115,101,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,118,97,114,32,111,116,104,101,114,95,112,114,111,116,111,99,111,108,32,61,32,106,115,111,110,46,115,99,104,101,109,101,115,91,48,93,59,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,106,115,111,110,46,115,99,104,101,109,101,115,91,48,93,32,61,32,99,117,114,114,101,110,116,95,112,114,111,116,111,99,111,108,59,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,106,115,111,110,46,115,99,104,101,109,101,115,91,49,93,32,61,32,111,116,104,101,114,95,112,114,111,116,111,99,111,108,59,10,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,125,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,106,115,111,110,46,104,111,115,116,32,61,32,119,105,110,100,111,119,46,108,111,99,97,116,105,111,110,46,104,111,115,116,59,32,32,47,47,32,115,101,116,115,32,116,104,101,32,99,117,114,114,101,110,116,32,104,111,115,116,10,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,99,111,110,115,116,32,117,105,32,61,32,83,119,97,103,103,101,114,85,73,66,117,110,100,108,101,40,123,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,115,112,101,99,58,32,106,115,111,110,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,118,97,108,105,100,97,116,111,114,85,114,108,58,32,110,117,108,108,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,100,111,109,95,105,100,58,32,39,35,115,119,97,103,103,101,114,45,117,105,39,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,100,101,101,112,76,105,110,107,105,110,103,58,32,116,114,117,101,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,106,115,111,110,69,100,105,116,111,114,58,32,116,114,117,101,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,100,111,99,69,120,112,97,110,115,105,111,110,58,32,34,110,111,110,101,34,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,97,112,105,115,83,111,114,116,101,114,58,32,34,97,108,112,104,97,34,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,47,47,111,112,101,114,97,116,105,111,110,115,83,111,114,116,101,114,58,32,34,97,108,112,104,97,34,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,112,114,101,115,101,116,115,58,32,91,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,83,119,97,103,103,101,114,85,73,66,117,110,100,108,101,46,112,114,101,115,101,116,115,46,97,112,105,115,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,47,47,32,121,97,121,32,69,83,54,32,109,111,100,117,108,101,115,32,226,134,152,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,65,114,114,97,121,46,105,115,65,114,114,97,121,40,83,119,97,103,103,101,114,85,73,83,116,97,110,100,97,108,111,110,101,80,114,101,115,101,116,41,32,63,32,83,119,97,103,103,101,114,85,73,83,116,97,110,100,97,108,111,110,101,80,114,101,115,101,116,32,58,32,83,119,97,103,103,101,114,85,73,83,116,97,110,100,97,108,111,110,101,80,114,101,115,101,116,46,100,101,102,97,117,108,116,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,93,44,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,112,108,117,103,105,110,115,58,32,91,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,83,119,97,103,103,101,114,85,73,66,117,110,100,108,101,46,112,108,117,103,105,110,115,46,68,111,119,110,108,111,97,100,85,114,108,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,93,44,10,32,32,32,32,32,32,32,32,32,32,32,32,10,32,32,32,32,32,32,32,32,32,32,32,32,47,47,32,108,97,121,111,117,116,58,32,34,83,116,97,110,100,97,108,111,110,101,76,97,121,111,117,116,34,32,32,47,47,32,117,110,99,111,109,109,101,110,116,32,116,111,32,101,110,97,98,108,101,32,116,104,101,32,103,114,101,101,110,32,116,111,112,32,104,101,97,100,101,114,10,32,32,32,32,32,32,32,32,125,41,10,10,32,32,32,32,32,32,32,32,119,105,110,100,111,119,46,117,105,32,61,32,117,105,10,10,32,32,32,32,32,32,32,32,47,47,32,117,110,99,111,109,109,101,110,116,32,116,111,32,114,101,110,97,109,101,32,116,104,101,32,116,111,112,32,98,114,97,110,100,32,105,102,32,108,97,121,111,117,116,32,105,115,32,101,110,97,98,108,101,100,10,32,32,32,32,32,32,32,32,47,47,32,36,40,34,46,116,111,112,98,97,114,45,119,114,97,112,112,101,114,32,46,108,105,110,107,32,115,112,97,110,34,41,46,114,101,112,108,97,99,101,87,105,116,104,40,34,60,115,112,97,110,62,104,116,116,112,98,105,110,60,47,115,112,97,110,62,34,41,59,10,32,32,32,32,32,32,32,32,125,41,10,32,32,32,32,125,41,10,125,10,32,32,32,32,60,47,115,99,114,105,112,116,62,32,32,60,100,105,118,32,99,108,97,115,115,61,39,115,119,97,103,103,101,114,45,117,105,39,62,10,32,32,32,32,60,100,105,118,32,99,108,97,115,115,61,34,119,114,97,112,112,101,114,34,62,10,32,32,32,32,32,32,32,32,60,115,101,99,116,105,111,110,32,99,108,97,115,115,61,34,98,108,111,99,107,32,99,111,108,45,49,50,32,98,108,111,99,107,45,100,101,115,107,116,111,112,32,99,111,108,45,49,50,45,100,101,115,107,116,111,112,34,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,100,105,118,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,104,50,62,79,116,104,101,114,32,85,116,105,108,105,116,105,101,115,60,47,104,50,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,117,108,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,108,105,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,97,32,104,114,101,102,61,34,47,102,111,114,109,115,47,112,111,115,116,34,62,72,84,77,76,32,102,111,114,109,60,47,97,62,32,116,104,97,116,32,112,111,115,116,115,32,116,111,32,47,112,111,115,116,32,47,102,111,114,109,115,47,112,111,115,116,60,47,108,105,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,47,117,108,62,10,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,98,114,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,60,98,114,32,47,62,10,32,32,32,32,32,32,32,32,32,32,32,32,60,47,100,105,118,62,10,32,32,32,32,32,32,32,32,60,47,115,101,99,116,105,111,110,62,10,32,32,32,32,60,47,100,105,118,62,10,60,47,100,105,118,62,10,60,47,98,111,100,121,62,10,10,60,47,104,116,109,108,62]) return {smallUint8, bigUint8} } ```
ronag commented 3 years ago

@jasnell

jasnell commented 3 years ago

Yeah, it was implemented to be spec compliant and not really to be performant. I'm sure there is lots of room for improvement. I'd love to see someone take on that work.

targos commented 3 years ago

What I found in the mentioned issue is that the time is mostly spent in this function:

https://github.com/nodejs/node/blob/e2148d742549b7c88891bb50b4b4a562bf4bd46a/src/node_i18n.cc#L432

XadillaX commented 3 years ago

In my machine, the test code ran in Chrome results in: 49 and 834ms. But Node.js results in 113 and 1730ms.

When I add two lines in ConverterObject just after `ucnv_toUnicode',

args.GetReturnValue().Set(Buffer::New(env, 0).ToLocalChecked());
return;

That means DO NOT do the left logic after decoding via ucnv_toUnicode. It results in 73 and 933ms. I think the left logic is not that fast.

XadillaX commented 3 years ago

btw, it seems ret = ToBufferEndian(env, &result); is very slow.

uasan commented 3 years ago

All I/O optimizations are multiplied by zero, when decoding text, this main cases for clients - Redis, PostgreSQL, etc...

liuxingbaoyu commented 2 years ago

It seems https://github.com/nodejs/node/pull/42695 fixes this issue.

node

PS F:\node> F:\node\out\release\node --expose_gc 1\1.js                                                                                                                                      small TextDe decode: 209.075ms
big TextDe decode: 1.765s

PS F:\node> node --expose_gc 1\1.js                    
small TextDe decode: 170.5ms
big TextDe decode: 3.058s

chrome

small TextDe decode: 75.279052734375 ms
big TextDe decode: 1370.48095703125 ms
anonrig commented 1 year ago

Here's the output before after 2 pull requests I opened on TextDecoder:

Before (v18.11.0):

small TextDe decode: 51.216ms
big TextDe decode: 924.02ms

After (Main):

small TextDe decode: 23.65ms
big TextDe decode: 773.155ms
anonrig commented 1 year ago

Bun:

➜  node-test bun ./text-decoder.js
[23.51ms] small TextDe decode
[129.33ms] big TextDe decode

Deno:

➜  node-test deno run text-decoder.js
small TextDe decode: 50ms
big TextDe decode: 439ms
anonrig commented 1 year ago

@srl295 do you have any suggestions for optimizations on large values?

jasnell commented 1 year ago

@srl295 ... as a reminder for context... this is making use of the ucnv_toUnicode API. We don't preflight anything, just allocate a large enough space, call ucnv_toUnicode, and process the result into a v8 string.