valyala / fastjson

Fast JSON parser and validator for Go. No custom structs, no code generation, no reflection
MIT License
2.3k stars 138 forks source link

Add ValidParser #68

Open kevburnsjr opened 3 years ago

kevburnsjr commented 3 years ago

Presently, users that want to validate their json before parsing need to run Validate and Parse separately.
This is inefficient because each json string must be scanned twice.

This PR introduces a ValidParser which validates as it parses.

It is slightly less efficient than the normal parser, but much more efficient than running Validate and Parse separately.

BenchmarkParse/small/fastjson-12                        10067671      111.0 ns/op  1712.14 MB/s    0 B/op  0 allocs/op
BenchmarkParse/small/fastjson-valid-parser-12           10503465      109.8 ns/op  1729.86 MB/s    0 B/op  0 allocs/op
BenchmarkParse/small/fastjson-validate-and-parse-12      6553486      176.5 ns/op  1076.23 MB/s    0 B/op  0 allocs/op

BenchmarkParse/medium/fastjson-12                        1797662      639.2 ns/op  3643.68 MB/s    0 B/op  0 allocs/op
BenchmarkParse/medium/fastjson-valid-parser-12           1635453      795.7 ns/op  2927.14 MB/s    0 B/op  0 allocs/op
BenchmarkParse/medium/fastjson-validate-and-parse-12      945916     1400 ns/op    1663.97 MB/s    0 B/op  0 allocs/op

BenchmarkParse/large/fastjson-12                          121093     9001 ns/op    3123.79 MB/s    0 B/op  0 allocs/op
BenchmarkParse/large/fastjson-valid-parser-12             116598    10946 ns/op    2568.69 MB/s    0 B/op  0 allocs/op
BenchmarkParse/large/fastjson-validate-and-parse-12        57669    20630 ns/op    1362.95 MB/s    0 B/op  0 allocs/op

BenchmarkParse/canada/fastjson-12                           1072  1205458 ns/op    1867.39 MB/s    1 B/op  0 allocs/op
BenchmarkParse/canada/fastjson-valid-parser-12              1022  1151216 ns/op    1955.38 MB/s    1 B/op  0 allocs/op
BenchmarkParse/canada/fastjson-validate-and-parse-12         678  1781026 ns/op    1263.91 MB/s    2 B/op  0 allocs/op

BenchmarkParse/citm/fastjson-12                             2584   476059 ns/op    3628.13 MB/s    0 B/op  0 allocs/op
BenchmarkParse/citm/fastjson-valid-parser-12                2528   481104 ns/op    3590.09 MB/s    0 B/op  0 allocs/op
BenchmarkParse/citm/fastjson-validate-and-parse-12          1443   844153 ns/op    2046.08 MB/s    1 B/op  0 allocs/op

BenchmarkParse/twitter/fastjson-12                          6711   164653 ns/op    3835.42 MB/s  756 B/op  0 allocs/op
BenchmarkParse/twitter/fastjson-valid-parser-12             5918   192253 ns/op    3284.80 MB/s    0 B/op  0 allocs/op
BenchmarkParse/twitter/fastjson-validate-and-parse-12       3758   317174 ns/op    1991.06 MB/s    0 B/op  0 allocs/op

Adds 7 new symbols to the API:

type ValidScanner
type ValidParser 
type ValidParserPool
func ValidParse(s string) (*Value, error)
func ValidParseBytes(b []byte) (*Value, error)
func MustValidParse(s string) *Value
func MustValidParseBytes(b []byte) *Value

See #58

codecov[bot] commented 3 years ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6dae91c) 97.11% compared to head (80b9537) 97.77%. Report is 2 commits behind head on master.

:exclamation: Current head 80b9537 differs from pull request most recent head f240cbf. Consider uploading reports for the commit f240cbf to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #68 +/- ## ========================================== + Coverage 97.11% 97.77% +0.66% ========================================== Files 9 11 +2 Lines 1073 1306 +233 ========================================== + Hits 1042 1277 +235 + Misses 22 21 -1 + Partials 9 8 -1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

flowchartsman commented 1 year ago

I, too, do not enjoy needing two calls for peace-of-mind that my JSON will not break once I start processing it. This would be super handy to have merged into the library, and the PR looks well-tested and benchmarked.

@valyala, is there some hesitation with this feature which has left it in limbo for so long? If not, I think a lot of people would appreciate being able to avoid separate invocations. 🙏

kevburnsjr commented 9 months ago

I've updated the symbol names. ValidateParser is now ValidParser for brevity. I will maintain a fork with this PR merged until this PR is merged for anyone who wants to use it. https://github.com/logbn/fastjson-valid