search

draivin / XNBNode

45 stars 14 forks source link

Truncate Chinese string in pack operation #5

Closed leozzyzheng closed 7 years ago

leozzyzheng commented 7 years ago

In writer.js:

class StringWriter {
    write(buffer, text, writerResolver) {
        let stringBuffer = new Buffer(text.length * 2);  // <- this line have a problem
        let size = stringBuffer.write(text);
        buffer.write7BitEncodedNumber(size);
        buffer.concat(stringBuffer.slice(0, size));
    }
}

A Chinese character would has 3 or 4 bytes length, so 2 will cause the string truncated. I change it to 4 and seems works well, but there may be more than 4 bytes length characters in some languages.

draivin commented 7 years ago

Good night (or morning over there), as you seem to have noticed, utf-8 will have at most 4 bytes per character, so changing it to * 4 will be enough, thanks for the report!

draivin commented 7 years ago

Fixed by this commit!

draivin commented 7 years ago

Hmm, things don't seem to be so simple, as the test fails when changing the * 2 to * 4 (the tests fail with alternate languages to begin with), so I'll be reopening this issue for now.

draivin commented 7 years ago

It seems I got ahead of myself and reopened this issue based on first pass errors, which seem to be expected (though I didn't remember that, it's been a while since I've messed with xnb-node internals), so I'm closing this issue again, sorry for the trouble.

leozzyzheng commented 7 years ago

No worries glad to see it helps :)