ibireme / yyjson

The fastest JSON library in C
https://ibireme.github.io/yyjson/doc/doxygen/html/
MIT License
3.04k stars 262 forks source link

Add option to allow writing invalid unicode #79

Closed pavelxdd closed 2 years ago

pavelxdd commented 2 years ago

Add a new option YYJSON_WRITE_ALLOW_INVALID_UNICODE (optional via YYJSON_DISABLE_NON_STANDARD) to allow parsing invalid unicode data.

When YYJSON_WRITE_ESCAPE_UNICODE flag is set don't try to parse malformed unicode sequences, but copy them per-byte instead.

codecov[bot] commented 2 years ago

Codecov Report

Merging #79 (a867d02) into master (c2df748) will decrease coverage by 0.62%. The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master      #79      +/-   ##
==========================================
- Coverage   97.68%   97.05%   -0.63%     
==========================================
  Files           2        2              
  Lines        2158     2172      +14     
==========================================
  Hits         2108     2108              
- Misses         50       64      +14     
Impacted Files Coverage Δ
src/yyjson.c 97.30% <0.00%> (-0.68%) :arrow_down:
src/yyjson.h 93.28% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c2df748...a867d02. Read the comment docs.

pavelxdd commented 2 years ago

Just checking length is not enough, there needs to be a proper unicode validation, like you showed in example here: https://github.com/ibireme/yyjson/pull/78#issuecomment-1104533145

pavelxdd commented 2 years ago

I've implemented unicode validation based on read_string to use it in my project now https://github.com/pavelxdd/yyjson/commit/2ba4a47e4412340edb38a6ea75f218c1b34ed6ac

At least it works for simple tests, including "\xF0abc".