Closed Max-Clark closed 1 year ago
Hey - thanks for a great investigation and write-up of this problem! I haven't touched this project in awhile - I've been working on some other stuff. Great sleuthing to figure out the problem.
I like the fix, but it looks like your PR has some random other changes mixed in too (adding an npm package.lock file, updating some dependencies, etc). I'll take a look...
Thanks for checking in! I'm happy to revert the other changes, just let me know.
Oh - thanks for the offer but I've just cherry-picked the patches out of your PR.
Fixed and published as 1.1.4. Thanks for your PR! :D
Description
While testing, we ran into an intermittent issue with the default JSON encoder,
Unexpected token \u0000 in JSON at position 1023
. JSON stringified payloads that were length 1024 were being saved with\u0000
as the last character verified by fdbcli.The issue appears to be an off-by-one error In the toStringParams function. To leave space for the null termination, the
napi_get_value_string_utf8
function writes up tobufsize - 1
and sets the byte after the last written to\0
shown here. If a string length 1024 enters theif (!buf_in_use && result->len <= sizeof(sp_buf))
block, only 1023 bytes will be written, and index 1023 will be written as\0
. Any string length less than 1024 does not have this problem, as it always has space in the buffer, and any string length greater than 1024 goes to the other block.The simplest and safest way appeared to be reducing the if block to
< sizeof(sp_buf)
.As far as I could tell, only the json encoder sends the value by napi_string with the built-in encoders, so that's the only test I added.
Type of change
How Has This Been Tested?
An addition to ks.ts adds coverage to this. npm tests passed with my machine's configuration.
Test Configuration: FoundationDB version 7.1.27 running in docker in host mode Ubuntu 20.04 Node 18.14.0 LTS
Checklist:
Thanks for your work!