benbjohnson / megajson

A JSON parser generator for high performance encoding and decoding in Go.
MIT License
466 stars 36 forks source link

Add support long strings #15

Closed aybabtme closed 10 years ago

aybabtme commented 10 years ago

This is not perfect and I'm sure a better way could be found. It negatively affect performance on strings:

master:

BenchmarkScanNumber 50000000          58.6 ns/op    51.23 MB/s
BenchmarkScanString 20000000          98.4 ns/op   101.58 MB/s
BenchmarkScanLongString  5000000       489 ns/op   116.42 MB/s
BenchmarkScanEscapedString  10000000   149 ns/op   120.50 MB/s
BenchmarkReadString 10000000           153 ns/op    65.04 MB/s
BenchmarkReadLongString  5000000       557 ns/op   102.18 MB/s
BenchmarkReadInt  50000000            62.5 ns/op    79.99 MB/s
BenchmarkReadFloat64  10000000         163 ns/op    97.89 MB/s
BenchmarkReadBool 50000000            62.0 ns/op    64.52 MB/s
ok    github.com/benbjohnson/megajson/scanner 23.022s

This branch:

BenchmarkScanNumber 50000000          54.2 ns/op    55.36 MB/s
BenchmarkScanString 20000000           105 ns/op    94.56 MB/s
BenchmarkScanLongString  5000000       540 ns/op   105.52 MB/s
BenchmarkScanEscapedString  10000000   153 ns/op   117.46 MB/s
BenchmarkReadString 10000000           153 ns/op    65.02 MB/s
BenchmarkReadLongString  5000000       598 ns/op    95.16 MB/s
BenchmarkReadInt  50000000            64.5 ns/op    77.50 MB/s
BenchmarkReadFloat64  10000000         175 ns/op    91.14 MB/s
BenchmarkReadBool 50000000            60.6 ns/op    66.05 MB/s
ok    github.com/aybabtme/megajson/scanner  23.710s

I think this is a hack and it's not very clean. I'd like to have input as to how this could be done better. Should there be another func that is delegated to once n == len(s.scratch)?

I'll try that out next.

benbjohnson commented 10 years ago

Besides the append() calls it looks good to me. Sorry for the delay in reviewing it.

aybabtme commented 10 years ago

Before using append(dst, src...):

BenchmarkScanNumber 50000000          54.3 ns/op    55.25 MB/s
BenchmarkScanString 20000000           110 ns/op    90.27 MB/s
BenchmarkScanLongString  5000000       599 ns/op    95.15 MB/s
BenchmarkScanEscapedString  10000000   166 ns/op   108.13 MB/s
BenchmarkReadString 10000000           162 ns/op    61.69 MB/s
BenchmarkReadLongString  5000000       661 ns/op    86.11 MB/s
BenchmarkReadInt  20000000            76.9 ns/op    65.06 MB/s
BenchmarkReadFloat64  10000000         188 ns/op    84.80 MB/s
BenchmarkReadBool 50000000            63.6 ns/op    62.94 MB/s

After:

BenchmarkScanNumber 50000000          55.3 ns/op    54.27 MB/s
BenchmarkScanString 20000000           109 ns/op    91.71 MB/s
BenchmarkScanLongString  5000000       561 ns/op   101.57 MB/s
BenchmarkScanEscapedString  10000000   152 ns/op   118.31 MB/s
BenchmarkReadString 10000000           158 ns/op    62.99 MB/s
BenchmarkReadLongString  5000000       619 ns/op    92.07 MB/s
BenchmarkReadInt  50000000            65.2 ns/op    76.70 MB/s
BenchmarkReadFloat64  10000000         179 ns/op    89.03 MB/s
BenchmarkReadBool 50000000            60.5 ns/op    66.14 MB/s
benbjohnson commented 10 years ago

:+1:

aybabtme commented 10 years ago

yay!