SIMD instruction can cause seg-fault/UMR

[GoogleCodeExporter]

The usage of rapidjson::SkipWhitespace_SIMD(char const*) reads 16 bytes 
(correct me if I'm wrong) at a time. It can cause uninitialized memory read on 
the following condition:
If rapidjson::SkipWhitespace_SIMD(char const*) is called at close to the end of 
string buffer which has less than 16 bytes of allocated space, the function 
will read beyond the memory it owns.

What steps will reproduce the problem?
Non-deterministic. However, you can get the error message just by running the 
simple test driver with valgrind.

What is the expected output? What do you see instead?
Can cause seg-fault. 

What version of the product are you using? On what operating system?
rhel5_x86-64, rhel6_x86-64

Original issue reported on code.google.com by lib...@gmail.com on 18 Mar 2014 at 10:57

[GoogleCodeExporter]

Original comment by milo...@gmail.com on 20 Jun 2014 at 11:21

• Changed state: Accepted

[GoogleCodeExporter]

I have inspected the code and tried to find solution.
However, I cannot find a possible workaround within the library.
This is because it will need the length of buffer in order to prevent reading 
out of boundary.
I think the easy and efficient way is to allocate extra 15 bytes in the source 
string buffer at user's code.
I can add some warning text in the document.
What do you think?

Original comment by milo...@gmail.com on 24 Jun 2014 at 2:52

• Changed state: Started

[GoogleCodeExporter]

That is the same workaround we use in our code. We had some ideas to fix it
(like take an extra string length argument) and conclude that it is too
much work comparing to just allocate 15 bytes more. I think it is fair to
do the workaround with proper warnings.
In our use case, we parse around 50 million JSON files/buffers per day and
we got hit by the bug around 100 times per day on average before the
workaround.

Original comment by lib...@gmail.com on 24 Jun 2014 at 6:40

[GoogleCodeExporter]

https://github.com/miloyip/rapidjson/issues/85
This bug has been fixed.
You can just copy the two SkipWhitespace_SIMD() functions to your version if 
you do not want to upgrade RapidJSON.
Please help verifying the bug if possible.
Thank you.

Original comment by milo...@gmail.com on 29 Jul 2014 at 9:00

• Changed state: Fixed

[GoogleCodeExporter]

Thanks. We are merging with your code and will profile with valgrind.

Original comment by lib...@gmail.com on 29 Jul 2014 at 5:59

[GoogleCodeExporter]

valgrind still complains for uninitialized memory read. I will send more
detailed analysis later.

Original comment by lib...@gmail.com on 30 Jul 2014 at 1:08