bytedance / sonic

A blazingly fast JSON serializing & deserializing library
Apache License 2.0
6.59k stars 327 forks source link

fix: always copy once after top get #577

Closed AsterDY closed 6 months ago

AsterDY commented 6 months ago

Background

sonic.Get() does copy the entired JSON and refer it to returned partial json. This behavior is neither CPU-friendly nor memory-safe: Copy entire JSON is time consuming, and returned JSON consumes more memory especially if it is cached or hold for a long term. Thus, We decide to not copy input JSON bytes, but copy returned partial JSON instead.

What's the effect

After this change, most normal services may gain a little CPU improve on sonic.Get(), and cache-like services will be less possible to be OOM. Only problem is for services who exists data-race on input JSON bytes, the panic may be more frequent. However, we believe this is a rare case in practice, and we DONOT expect any data race on input JSON.

codecov-commenter commented 6 months ago

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (63f4c07) 78.57% compared to head (ae7e496) 74.26%.

Files Patch % Lines
ast/search.go 70.83% 4 Missing and 3 partials :warning:
api.go 60.00% 2 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #577 +/- ## ========================================== - Coverage 78.57% 74.26% -4.32% ========================================== Files 69 63 -6 Lines 10823 8435 -2388 ========================================== - Hits 8504 6264 -2240 + Misses 1942 1830 -112 + Partials 377 341 -36 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.