search

go-json-experiment / json

Experimental implementation of a proposed v2 encoding/json package
BSD 3-Clause "New" or "Revised" License
377 stars 12 forks source link

Single line pretty-printing options #20

Closed veqryn closed 5 months ago

veqryn commented 11 months ago

Discussion: In the same vein as the WithIndent and WithIndentPrefix formatting options, I would really like the option to make json pretty but on a single line. I end up reading, or at least skimming, through a lot of new-line delimited json, and this would make my life better 100%.

Consider reading the following:

{"foo": "bar", "baz": 123, "qux": {"quux": "corge"}, "grault": {"garply": ["waldo", "fred"]}, "thud": "plugh"}
{"foo":"bar", "baz":123, "qux":{"quux":"corge"}, "grault":{"garply":["waldo", "fred"]}, "thud":"plugh"}
{"foo":"bar","baz":123,"qux":{"quux":"corge"},"grault":{"garply":["waldo","fred"]},"thud":"plugh"}

The first two are much easier to see the key-value pairs and the elements than the third one, which is the only option right now.

https://github.com/golang/go/discussions/63397#discussioncomment-7300039

PR Details: This PR adds two new boolean flag options: SpaceAfterColon and SpaceAfterComma I do like those to be separate options, as I prefer to have the space after comma but not after colon, while lots of other people like space after both. I'm not married to the naming or the implementation, and am open to any suggestions. Please let me know if I need to add any additional tests beyond the two I added.

dsnet commented 11 months ago

Thanks for the PR.

I apologize for being a stickler for legal rules. Since the goal of this is intended to merge into the Go toolchain, we'll need every contribution to have also signed the Go CLA. Is there a merged commit in any of the Go repositories that you can point to that proves you've signed the CLA? Thanks!

High-level thought: Modifying MayAppendDelim likely makes it such that it is no longer inlineable. We don't want to slow down the case where whitespace is completely elided. I'd like to see benchmarks of the performance delta of compacted output.

veqryn commented 11 months ago

Of course. I have signed the Go CLA. Here is my most recent contribution, though small: https://github.com/golang/go/commit/65c53a1833a26467357b4aa6223e4dde5d6d7ed0 https://go-review.googlesource.com/c/go/+/524618

veqryn commented 11 months ago

I am also curious on how it may affect performance. Do you want me to run benchmarks, and do those benchmarks already exist? Or is this something you'd like to do to keep consistency with previous runs?

veqryn commented 10 months ago

Ran some benchmarks, comparing against master. These runs are on a M1 macbook.

$ go env
GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/x/Library/Caches/go-build'
GOENV='/Users/x/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/x/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/cduncan/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/opt/go/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/opt/go/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.21.4'
GCCGO='gccgo'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/jx/04vrv3h53d72ylmj73g8r8lc0000gn/T/go-build682745689=/tmp/go-build -gno-record-gcc-switches -fno-common'

Master:

BenchmarkMarshal
BenchmarkMarshal/Bool
BenchmarkMarshal/Bool-10            17499763            68.53 ns/op   58.37 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/String
BenchmarkMarshal/String-10          16208367            73.79 ns/op  203.27 MB/s          16 B/op          1 allocs/op
BenchmarkMarshal/Int
BenchmarkMarshal/Int-10             17451297            68.37 ns/op   73.14 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Uint
BenchmarkMarshal/Uint-10            17472674            68.26 ns/op   58.60 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Float
BenchmarkMarshal/Float-10           11057541           107.5 ns/op    46.53 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Map/ManyEmpty
BenchmarkMarshal/Map/ManyEmpty-10             769255          1566 ns/op     192.16 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Map/OneLarge
BenchmarkMarshal/Map/OneLarge-10              635246          1877 ns/op     111.37 MB/s         384 B/op          4 allocs/op
BenchmarkMarshal/Map/ManySmall
BenchmarkMarshal/Map/ManySmall-10             177566          6728 ns/op      54.25 MB/s        4697 B/op         82 allocs/op
BenchmarkMarshal/Struct/ManyEmpty
BenchmarkMarshal/Struct/ManyEmpty-10          475663          2532 ns/op     118.86 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Struct/OneLarge
BenchmarkMarshal/Struct/OneLarge-10          1435818           835.9 ns/op   250.04 MB/s         224 B/op          1 allocs/op
BenchmarkMarshal/Struct/ManySmall
BenchmarkMarshal/Struct/ManySmall-10          635452          1892 ns/op     192.90 MB/s         384 B/op          1 allocs/op
BenchmarkMarshal/Slice/ManyEmpty
BenchmarkMarshal/Slice/ManyEmpty-10          1485180           796.3 ns/op   378.01 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Slice/OneLarge
BenchmarkMarshal/Slice/OneLarge-10           2681997           446.9 ns/op   234.94 MB/s         112 B/op          1 allocs/op
BenchmarkMarshal/Slice/ManySmall
BenchmarkMarshal/Slice/ManySmall-10          1233034           962.5 ns/op   163.11 MB/s         160 B/op          1 allocs/op
BenchmarkMarshal/Array/OneLarge
BenchmarkMarshal/Array/OneLarge-10           2692856           449.6 ns/op   233.55 MB/s         112 B/op          1 allocs/op
BenchmarkMarshal/Array/ManySmall
BenchmarkMarshal/Array/ManySmall-10          1332310           899.0 ns/op   174.64 MB/s         160 B/op          1 allocs/op
BenchmarkMarshal/Bytes/Slice
BenchmarkMarshal/Bytes/Slice-10              8157802           147.0 ns/op   312.87 MB/s         112 B/op          2 allocs/op
BenchmarkMarshal/Bytes/Array
BenchmarkMarshal/Bytes/Array-10              8151314           149.2 ns/op   308.39 MB/s         112 B/op          2 allocs/op
BenchmarkMarshal/Pointer
BenchmarkMarshal/Pointer-10                  7415541           161.8 ns/op    24.73 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/TextArshal
BenchmarkMarshal/TextArshal-10              13136809            90.71 ns/op   88.20 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/JSONArshalV1
BenchmarkMarshal/JSONArshalV1-10            13365788            89.04 ns/op   89.84 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/JSONArshalV2
BenchmarkMarshal/JSONArshalV2-10            13210863            90.46 ns/op   88.44 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Duration
BenchmarkMarshal/Duration-10                12109178            98.77 ns/op   80.99 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/Time
BenchmarkMarshal/Time-10                    11445398           104.4 ns/op   210.63 MB/s          24 B/op          1 allocs/op

BenchmarkTextValue
BenchmarkTextValue/IsValid
BenchmarkTextValue/IsValid-10                902       1216821 ns/op    1419.44 MB/s       23065 B/op        213 allocs/op
BenchmarkTextValue/Compact
BenchmarkTextValue/Compact-10               1080       1106798 ns/op    1560.54 MB/s           1 B/op          0 allocs/op
BenchmarkTextValue/Compact/Noop
BenchmarkTextValue/Compact/Noop-10          1815        655949 ns/op    2633.14 MB/s        1399 B/op          0 allocs/op
BenchmarkTextValue/Indent
BenchmarkTextValue/Indent-10                 648       1839437 ns/op     938.98 MB/s           1 B/op          0 allocs/op
BenchmarkTextValue/Indent/Noop
BenchmarkTextValue/Indent/Noop-10            642       1849534 ns/op     933.86 MB/s           2 B/op          0 allocs/op
BenchmarkTextValue/Indent#01
BenchmarkTextValue/Indent#01-10              394       3027398 ns/op     570.52 MB/s       29646 B/op        212 allocs/op
BenchmarkTextValue/Indent/Noop#01
BenchmarkTextValue/Indent/Noop#01-10         460       2598857 ns/op     664.60 MB/s       28752 B/op        212 allocs/op
BenchmarkTextValue/Canonicalize
BenchmarkTextValue/Canonicalize-10           393       3034790 ns/op     569.13 MB/s       29717 B/op        212 allocs/op
BenchmarkTextValue/Canonicalize/Noop
BenchmarkTextValue/Canonicalize/Noop-10                  460       2594072 ns/op     665.83 MB/s       28764 B/op        212 allocs/op

Single line pretty printing PR:

BenchmarkMarshal
BenchmarkMarshal/Bool
BenchmarkMarshal/Bool-10            15141565            69.03 ns/op   57.95 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/String
BenchmarkMarshal/String-10          15740070            75.42 ns/op  198.90 MB/s          16 B/op          1 allocs/op
BenchmarkMarshal/Int
BenchmarkMarshal/Int-10             17256627            68.17 ns/op   73.34 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Uint
BenchmarkMarshal/Uint-10            17343348            68.25 ns/op   58.61 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Float
BenchmarkMarshal/Float-10           11074166           107.2 ns/op    46.64 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Map/ManyEmpty
BenchmarkMarshal/Map/ManyEmpty-10             669688          1753 ns/op     171.72 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Map/OneLarge
BenchmarkMarshal/Map/OneLarge-10              606732          1950 ns/op     107.19 MB/s         384 B/op          4 allocs/op
BenchmarkMarshal/Map/ManySmall
BenchmarkMarshal/Map/ManySmall-10             174262          6799 ns/op      53.68 MB/s        4697 B/op         82 allocs/op
BenchmarkMarshal/Struct/ManyEmpty
BenchmarkMarshal/Struct/ManyEmpty-10          458298          2619 ns/op     114.91 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Struct/OneLarge
BenchmarkMarshal/Struct/OneLarge-10          1331934           899.9 ns/op   232.25 MB/s         224 B/op          1 allocs/op
BenchmarkMarshal/Struct/ManySmall
BenchmarkMarshal/Struct/ManySmall-10          608961          1962 ns/op     186.06 MB/s         384 B/op          1 allocs/op
BenchmarkMarshal/Slice/ManyEmpty
BenchmarkMarshal/Slice/ManyEmpty-10          1433493           826.3 ns/op   364.27 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Slice/OneLarge
BenchmarkMarshal/Slice/OneLarge-10           2447815           489.7 ns/op   214.43 MB/s         112 B/op          1 allocs/op
BenchmarkMarshal/Slice/ManySmall
BenchmarkMarshal/Slice/ManySmall-10          1000000          1040 ns/op     150.99 MB/s         160 B/op          1 allocs/op
BenchmarkMarshal/Array/OneLarge
BenchmarkMarshal/Array/OneLarge-10           2460807           487.5 ns/op   215.37 MB/s         112 B/op          1 allocs/op
BenchmarkMarshal/Array/ManySmall
BenchmarkMarshal/Array/ManySmall-10          1251490           959.1 ns/op   163.69 MB/s         160 B/op          1 allocs/op
BenchmarkMarshal/Bytes/Slice
BenchmarkMarshal/Bytes/Slice-10              8173449           147.6 ns/op   311.74 MB/s         112 B/op          2 allocs/op
BenchmarkMarshal/Bytes/Array
BenchmarkMarshal/Bytes/Array-10              8078925           149.3 ns/op   308.14 MB/s         112 B/op          2 allocs/op
BenchmarkMarshal/Pointer
BenchmarkMarshal/Pointer-10                  7258795           163.2 ns/op    24.50 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/TextArshal
BenchmarkMarshal/TextArshal-10              12987001            91.84 ns/op   87.10 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/JSONArshalV1
BenchmarkMarshal/JSONArshalV1-10            13303111            89.76 ns/op   89.13 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/JSONArshalV2
BenchmarkMarshal/JSONArshalV2-10            13645311            87.32 ns/op   91.61 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Duration
BenchmarkMarshal/Duration-10                12055948            98.91 ns/op   80.88 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/Time
BenchmarkMarshal/Time-10                    11438097           104.7 ns/op   210.15 MB/s          24 B/op          1 allocs/op

BenchmarkTextValue
BenchmarkTextValue/IsValid
BenchmarkTextValue/IsValid-10                918       1216074 ns/op    1420.31 MB/s       23072 B/op        213 allocs/op
BenchmarkTextValue/Compact
BenchmarkTextValue/Compact-10               1072       1114277 ns/op    1550.07 MB/s        2369 B/op          0 allocs/op
BenchmarkTextValue/Compact/Noop
BenchmarkTextValue/Compact/Noop-10          1754        671224 ns/op    2573.22 MB/s        1448 B/op          0 allocs/op
BenchmarkTextValue/Indent
BenchmarkTextValue/Indent-10                 648       1873448 ns/op     921.94 MB/s       16348 B/op          0 allocs/op
BenchmarkTextValue/Indent/Noop
BenchmarkTextValue/Indent/Noop-10            642       1874352 ns/op     921.49 MB/s           2 B/op          0 allocs/op
BenchmarkTextValue/Indent#01
BenchmarkTextValue/Indent#01-10              393       3045207 ns/op     567.19 MB/s       23165 B/op        212 allocs/op
BenchmarkTextValue/Indent/Noop#01
BenchmarkTextValue/Indent/Noop#01-10         460       2593190 ns/op     666.05 MB/s       28766 B/op        212 allocs/op
BenchmarkTextValue/Canonicalize
BenchmarkTextValue/Canonicalize-10           393       3037923 ns/op     568.55 MB/s       29628 B/op        212 allocs/op
BenchmarkTextValue/Canonicalize/Noop
BenchmarkTextValue/Canonicalize/Noop-10                  460       2591278 ns/op     666.55 MB/s       23077 B/op        212 allocs/op

Comparison between master and this pr:

0.7% slower     BenchmarkMarshal/Bool
2.2% slower     BenchmarkMarshal/String
0.3% faster     BenchmarkMarshal/Int
0.0% faster     BenchmarkMarshal/Uint
0.3% faster     BenchmarkMarshal/Float
10.0% slower        BenchmarkMarshal/Map/ManyEmpty
3.7% slower     BenchmarkMarshal/Map/OneLarge
1.0% slower     BenchmarkMarshal/Map/ManySmall
3.3% slower     BenchmarkMarshal/Struct/ManyEmpty
7.1% slower     BenchmarkMarshal/Struct/OneLarge
3.6% slower     BenchmarkMarshal/Struct/ManySmall
3.6% slower     BenchmarkMarshal/Slice/ManyEmpty
8.7% slower     BenchmarkMarshal/Slice/OneLarge
7.5% slower     BenchmarkMarshal/Slice/ManySmall
7.8% slower     BenchmarkMarshal/Array/OneLarge
6.3% slower     BenchmarkMarshal/Array/ManySmall
0.4% slower     BenchmarkMarshal/Bytes/Slice
0.0% faster     BenchmarkMarshal/Bytes/Array
0.9% slower     BenchmarkMarshal/Pointer
1.2% slower     BenchmarkMarshal/TextArshal
0.8% slower     BenchmarkMarshal/JSONArshalV1
3.6% faster     BenchmarkMarshal/JSONArshalV2
0.1% slower     BenchmarkMarshal/Duration
0.2% slower     BenchmarkMarshal/Time

0.0% faster     BenchmarkTextValue/IsValid
0.6% slower     BenchmarkTextValue/Compact
2.3% slower     BenchmarkTextValue/Compact/Noop
1.8% slower     BenchmarkTextValue/Indent
1.3% slower     BenchmarkTextValue/Indent/Noop
0.6% slower     BenchmarkTextValue/Indent#01
0.2% faster     BenchmarkTextValue/Indent/Noop#01
0.1% slower     BenchmarkTextValue/Canonicalize
0.1% faster     BenchmarkTextValue/Canonicalize/Noop

@dsnet What do you think?

dsnet commented 10 months ago

Thanks for running the benchmarks, but up to 10% slowdown is a fairly non-trivial performance detriment. Especially since the benchmarks are intended to exercise the performance Go reflection, not the syntactic processor.

dsnet commented 10 months ago

If you're okay with it, I can take a stab at implementing this.

veqryn commented 10 months ago

Ok. So here is what I plan to change, please let me know what you think:

veqryn commented 10 months ago

I'm ok if you want to take a stab. I'd probably prefer to collaborate so I can learn from you (including if we want to throw all of this out and start over, with different flags, etc), but am ok with anything.

dsnet commented 10 months ago

Feel free to keep working on it. Your idea is similar to what I had in mind. Essentially, we want to change nothing about the old code path if Expand&SpaceAfterColon&SpaceAfterComma are all false.

You can probably do something like:

package jsonflags

// AnyWhitespace reports whether the encoded output might have any whitespace.
const AnyWhitespace = SpaceAfterColon | SpaceAfterComma | Expand
package json

if optimizeCommon && !xe.Flags.Get(jsonflags.AnyWhitespace) && !xe.Tokens.Last.NeedObjectName() {
    ...
}

I believe this should work since jsonflags.Flags.Get is semantically equivalent to reporting whether any of the bits are set, which allows us to check all three flags in a single operation.

We might want to rename Expand to Multiline, since Expand is an ambiguous option now with the introduction of expansion within a single line.

dsnet commented 10 months ago

We also probably want to have the semantic that use of Multiline or Indent or IndentPrefix implies SpaceAfterColon if not already set.

dsnet commented 10 months ago

Also for benchmarks, you probably want to run these instead:

go test -run=- -bench="Testdata//(Marshal|Encode)/"
veqryn commented 10 months ago

Master:

BenchmarkMarshal/Bool-10    17468782            68.35 ns/op   58.52 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/String-10          16201582            73.64 ns/op  203.70 MB/s          16 B/op          1 allocs/op
BenchmarkMarshal/Int-10             17379528            68.38 ns/op   73.12 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Uint-10            17042438            68.85 ns/op   58.10 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Float-10           11098650           107.5 ns/op    46.50 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Map/ManyEmpty-10     767757          1564 ns/op     192.41 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Map/OneLarge-10      626533          1895 ns/op     110.29 MB/s         384 B/op          4 allocs/op
BenchmarkMarshal/Map/ManySmall-10     174926          6757 ns/op      54.02 MB/s        4697 B/op         82 allocs/op
BenchmarkMarshal/Struct/ManyEmpty-10              476367          2557 ns/op     117.73 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Struct/OneLarge-10              1440249           832.4 ns/op   251.09 MB/s         224 B/op          1 allocs/op
BenchmarkMarshal/Struct/ManySmall-10              634903          1877 ns/op     194.43 MB/s         384 B/op          1 allocs/op
BenchmarkMarshal/Slice/ManyEmpty-10              1481580           796.4 ns/op   377.97 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Slice/OneLarge-10               2679693           447.2 ns/op   234.77 MB/s         112 B/op          1 allocs/op
BenchmarkMarshal/Slice/ManySmall-10              1246032           961.9 ns/op   163.22 MB/s         160 B/op          1 allocs/op
BenchmarkMarshal/Array/OneLarge-10               2691718           445.1 ns/op   235.90 MB/s         112 B/op          1 allocs/op
BenchmarkMarshal/Array/ManySmall-10              1313089           904.8 ns/op   173.52 MB/s         160 B/op          1 allocs/op
BenchmarkMarshal/Bytes/Slice-10                  8135280           148.0 ns/op   310.91 MB/s         112 B/op          2 allocs/op
BenchmarkMarshal/Bytes/Array-10                  8096805           149.4 ns/op   307.93 MB/s         112 B/op          2 allocs/op
BenchmarkMarshal/Pointer-10                      7230853           164.8 ns/op    24.27 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/TextArshal-10                  13068982            91.06 ns/op   87.86 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/JSONArshalV1-10                13155099            90.63 ns/op   88.27 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/JSONArshalV2-10                13515478            87.46 ns/op   91.47 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Duration-10                    12179206            97.91 ns/op   81.71 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/Time-10                        11404070           104.9 ns/op   209.81 MB/s          24 B/op          1 allocs/op
BenchmarkTextValue/IsValid-10                        981       1217373 ns/op    1418.80 MB/s       23071 B/op        213 allocs/op
BenchmarkTextValue/Compact-10                       1083       1106176 ns/op    1561.42 MB/s        2345 B/op          0 allocs/op
BenchmarkTextValue/Compact/Noop-10                  1814        653127 ns/op    2644.52 MB/s           0 B/op          0 allocs/op
BenchmarkTextValue/Indent-10                         648       1834621 ns/op     941.45 MB/s           1 B/op          0 allocs/op
BenchmarkTextValue/Indent/Noop-10                    645       1848691 ns/op     934.28 MB/s           1 B/op          0 allocs/op
BenchmarkTextValue/Canonicalize-10                   393       3037866 ns/op     568.56 MB/s       29613 B/op        212 allocs/op
BenchmarkTextValue/Canonicalize/Noop-10              462       2592450 ns/op     666.24 MB/s       23071 B/op        212 allocs/op
BenchmarkTestdata/CanadaGeometry/Marshal/Concrete-10                 981       1171681 ns/op     230.78 MB/s      278646 B/op          1 allocs/op
BenchmarkTestdata/CanadaGeometry/Marshal/Interface-10                946       1258706 ns/op     214.83 MB/s      278682 B/op          2 allocs/op
BenchmarkTestdata/CanadaGeometry/Encode/Token/Streaming-10          1036       1150617 ns/op     235.01 MB/s        9416 B/op         38 allocs/op
BenchmarkTestdata/CanadaGeometry/Encode/Token/Buffered-10           1042       1145659 ns/op     236.02 MB/s        1192 B/op         26 allocs/op
BenchmarkTestdata/CanadaGeometry/Encode/Value/Streaming-10          2500        465955 ns/op     580.32 MB/s     1187561 B/op         46 allocs/op
BenchmarkTestdata/CanadaGeometry/Encode/Value/Buffered-10           3492        342796 ns/op     788.82 MB/s        1024 B/op         20 allocs/op
BenchmarkTestdata/CitmCatalog/Marshal/Concrete-10                   1138       1045570 ns/op    1651.93 MB/s      511155 B/op        133 allocs/op
BenchmarkTestdata/CitmCatalog/Marshal/Interface-10                   645       1839433 ns/op     938.99 MB/s      508006 B/op          2 allocs/op
BenchmarkTestdata/CitmCatalog/Encode/Token/Streaming-10              686       1740098 ns/op     992.59 MB/s       36116 B/op        271 allocs/op
BenchmarkTestdata/CitmCatalog/Encode/Token/Buffered-10               694       1721999 ns/op    1003.02 MB/s       27823 B/op        258 allocs/op
BenchmarkTestdata/CitmCatalog/Encode/Value/Streaming-10              724       1638241 ns/op    1054.30 MB/s     2564896 B/op        271 allocs/op
BenchmarkTestdata/CitmCatalog/Encode/Value/Buffered-10               824       1448067 ns/op    1192.76 MB/s       26660 B/op        241 allocs/op
BenchmarkTestdata/GolangSource/Marshal/Concrete-10                   307       3904894 ns/op     496.93 MB/s     1941727 B/op          1 allocs/op
BenchmarkTestdata/GolangSource/Marshal/Interface-10                  193       6155516 ns/op     315.24 MB/s     1941700 B/op          2 allocs/op
BenchmarkTestdata/GolangSource/Encode/Token/Streaming-10             183       6505386 ns/op     298.29 MB/s       17776 B/op        168 allocs/op
BenchmarkTestdata/GolangSource/Encode/Token/Buffered-10              186       6448351 ns/op     300.93 MB/s        9344 B/op        153 allocs/op
BenchmarkTestdata/GolangSource/Encode/Value/Streaming-10             282       4234603 ns/op     458.24 MB/s    10599377 B/op        189 allocs/op
BenchmarkTestdata/GolangSource/Encode/Value/Buffered-10              337       3550961 ns/op     546.46 MB/s        8408 B/op        153 allocs/op
BenchmarkTestdata/StringEscaped/Marshal/Concrete-10                50049         24098 ns/op    1745.42 MB/s       18438 B/op          1 allocs/op
BenchmarkTestdata/StringEscaped/Marshal/Interface-10               40562         29717 ns/op    1415.43 MB/s       18449 B/op          2 allocs/op
BenchmarkTestdata/StringEscaped/Encode/Token/Streaming-10          37190         32134 ns/op    1308.97 MB/s       12624 B/op         29 allocs/op
BenchmarkTestdata/StringEscaped/Encode/Token/Buffered-10           38908         30716 ns/op    1369.38 MB/s        4080 B/op         19 allocs/op
BenchmarkTestdata/StringEscaped/Encode/Value/Streaming-10           4125        286689 ns/op     146.72 MB/s      151016 B/op        216 allocs/op
BenchmarkTestdata/StringEscaped/Encode/Value/Buffered-10            4305        278973 ns/op     150.77 MB/s       67632 B/op        201 allocs/op
BenchmarkTestdata/StringUnicode/Marshal/Concrete-10                49994         24071 ns/op     752.92 MB/s       18438 B/op          1 allocs/op
BenchmarkTestdata/StringUnicode/Marshal/Interface-10               40437         29758 ns/op     609.05 MB/s       18447 B/op          2 allocs/op
BenchmarkTestdata/StringUnicode/Encode/Token/Streaming-10          37274         32188 ns/op     563.07 MB/s       12624 B/op         29 allocs/op
BenchmarkTestdata/StringUnicode/Encode/Token/Buffered-10           38970         30717 ns/op     590.04 MB/s        4080 B/op         19 allocs/op
BenchmarkTestdata/StringUnicode/Encode/Value/Streaming-10          33397         35745 ns/op     507.04 MB/s       87456 B/op         33 allocs/op
BenchmarkTestdata/StringUnicode/Encode/Value/Buffered-10           42044         28522 ns/op     635.43 MB/s        4072 B/op         18 allocs/op
BenchmarkTestdata/SyntheaFhir/Marshal/Concrete-10                    178       6686264 ns/op     300.39 MB/s     3141766 B/op       2569 allocs/op
BenchmarkTestdata/SyntheaFhir/Marshal/Interface-10                   405       2936731 ns/op     683.92 MB/s     1146993 B/op          2 allocs/op
BenchmarkTestdata/SyntheaFhir/Encode/Token/Streaming-10              470       2538261 ns/op     791.29 MB/s       11832 B/op         77 allocs/op
BenchmarkTestdata/SyntheaFhir/Encode/Token/Buffered-10               480       2495617 ns/op     804.81 MB/s        3528 B/op         63 allocs/op
BenchmarkTestdata/SyntheaFhir/Encode/Value/Streaming-10              434       2736869 ns/op     733.87 MB/s     6646266 B/op         95 allocs/op
BenchmarkTestdata/SyntheaFhir/Encode/Value/Buffered-10               543       2219020 ns/op     905.13 MB/s        3840 B/op         61 allocs/op
BenchmarkTestdata/TwitterStatus/Marshal/Concrete-10                 1587        743084 ns/op     849.86 MB/s      501913 B/op         59 allocs/op
BenchmarkTestdata/TwitterStatus/Marshal/Interface-10                1170       1016499 ns/op     621.26 MB/s      467015 B/op          2 allocs/op
BenchmarkTestdata/TwitterStatus/Encode/Token/Streaming-10           1075       1111981 ns/op     567.92 MB/s       17464 B/op         94 allocs/op
BenchmarkTestdata/TwitterStatus/Encode/Token/Buffered-10            1096       1090856 ns/op     578.92 MB/s        9808 B/op         81 allocs/op
BenchmarkTestdata/TwitterStatus/Encode/Value/Streaming-10           1023       1156195 ns/op     546.20 MB/s     2551920 B/op        119 allocs/op
BenchmarkTestdata/TwitterStatus/Encode/Value/Buffered-10            1233        968893 ns/op     651.79 MB/s       13112 B/op         89 allocs/op

This PR:

BenchmarkMarshal/Bool-10    15258075            67.49 ns/op   59.26 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/String-10          16305501            73.41 ns/op  204.34 MB/s          16 B/op          1 allocs/op
BenchmarkMarshal/Int-10             17270784            70.04 ns/op   71.39 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Uint-10            17452978            68.22 ns/op   58.64 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Float-10           10972519           108.5 ns/op    46.10 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Map/ManyEmpty-10     738729          1627 ns/op     185.03 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Map/OneLarge-10      635596          1867 ns/op     111.97 MB/s         384 B/op          4 allocs/op
BenchmarkMarshal/Map/ManySmall-10     176860          6677 ns/op      54.67 MB/s        4697 B/op         82 allocs/op
BenchmarkMarshal/Struct/ManyEmpty-10              447837          2711 ns/op     111.04 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Struct/OneLarge-10              1396676           862.1 ns/op   242.44 MB/s         224 B/op          1 allocs/op
BenchmarkMarshal/Struct/ManySmall-10              625180          1918 ns/op     190.26 MB/s         384 B/op          1 allocs/op
BenchmarkMarshal/Slice/ManyEmpty-10              1448064           833.4 ns/op   361.15 MB/s         320 B/op          1 allocs/op
BenchmarkMarshal/Slice/OneLarge-10               2511039           479.5 ns/op   218.99 MB/s         112 B/op          1 allocs/op
BenchmarkMarshal/Slice/ManySmall-10              1000000          1019 ns/op     154.01 MB/s         160 B/op          1 allocs/op
BenchmarkMarshal/Array/OneLarge-10               2686700           445.5 ns/op   235.71 MB/s         112 B/op          1 allocs/op
BenchmarkMarshal/Array/ManySmall-10              1327821           903.9 ns/op   173.69 MB/s         160 B/op          1 allocs/op
BenchmarkMarshal/Bytes/Slice-10                  8212627           146.3 ns/op   314.47 MB/s         112 B/op          2 allocs/op
BenchmarkMarshal/Bytes/Array-10                  8089743           147.9 ns/op   311.04 MB/s         112 B/op          2 allocs/op
BenchmarkMarshal/Pointer-10                      7419316           160.4 ns/op    24.94 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/TextArshal-10                  13062153            91.14 ns/op   87.78 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/JSONArshalV1-10                13446426            88.40 ns/op   90.50 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/JSONArshalV2-10                13279116            90.40 ns/op   88.49 MB/s           8 B/op          1 allocs/op
BenchmarkMarshal/Duration-10                    12004356            99.55 ns/op   80.37 MB/s          16 B/op          2 allocs/op
BenchmarkMarshal/Time-10                        11433212           104.3 ns/op   210.88 MB/s          24 B/op          1 allocs/op
BenchmarkTextValue/IsValid-10                        979       1216533 ns/op    1419.78 MB/s       23075 B/op        213 allocs/op
BenchmarkTextValue/Compact-10                       1071       1096822 ns/op    1574.73 MB/s           1 B/op          0 allocs/op
BenchmarkTextValue/Compact/Noop-10                  1801        660279 ns/op    2615.87 MB/s           0 B/op          0 allocs/op
BenchmarkTextValue/Indent-10                         645       1848907 ns/op     934.18 MB/s           1 B/op          0 allocs/op
BenchmarkTextValue/Indent/Noop-10                    636       1874258 ns/op     921.54 MB/s           2 B/op          0 allocs/op
BenchmarkTextValue/Canonicalize-10                   394       3024818 ns/op     571.01 MB/s       29674 B/op        212 allocs/op
BenchmarkTextValue/Canonicalize/Noop-10              460       2592809 ns/op     666.15 MB/s       23054 B/op        212 allocs/op
BenchmarkTestdata/CanadaGeometry/Marshal/Concrete-10                 942       1173041 ns/op     230.51 MB/s      279907 B/op          1 allocs/op
BenchmarkTestdata/CanadaGeometry/Marshal/Interface-10                940       1268764 ns/op     213.12 MB/s      278682 B/op          2 allocs/op
BenchmarkTestdata/CanadaGeometry/Encode/Token/Streaming-10          1034       1153147 ns/op     234.49 MB/s        9416 B/op         38 allocs/op
BenchmarkTestdata/CanadaGeometry/Encode/Token/Buffered-10           1040       1150103 ns/op     235.11 MB/s        1192 B/op         26 allocs/op
BenchmarkTestdata/CanadaGeometry/Encode/Value/Streaming-10          2407        471761 ns/op     573.18 MB/s     1187561 B/op         46 allocs/op
BenchmarkTestdata/CanadaGeometry/Encode/Value/Buffered-10           3458        346602 ns/op     780.15 MB/s        1024 B/op         20 allocs/op
BenchmarkTestdata/CitmCatalog/Marshal/Concrete-10                   1104       1082243 ns/op    1595.95 MB/s      513456 B/op        133 allocs/op
BenchmarkTestdata/CitmCatalog/Marshal/Interface-10                   642       1851416 ns/op     932.91 MB/s      508000 B/op          2 allocs/op
BenchmarkTestdata/CitmCatalog/Encode/Token/Streaming-10              684       1745665 ns/op     989.42 MB/s       36109 B/op        271 allocs/op
BenchmarkTestdata/CitmCatalog/Encode/Token/Buffered-10               691       1729951 ns/op     998.41 MB/s       27813 B/op        258 allocs/op
BenchmarkTestdata/CitmCatalog/Encode/Value/Streaming-10              717       1655998 ns/op    1043.00 MB/s     2564897 B/op        271 allocs/op
BenchmarkTestdata/CitmCatalog/Encode/Value/Buffered-10               819       1458285 ns/op    1184.41 MB/s       26637 B/op        241 allocs/op
BenchmarkTestdata/GolangSource/Marshal/Concrete-10                   298       4016621 ns/op     483.11 MB/s     2012869 B/op          1 allocs/op
BenchmarkTestdata/GolangSource/Marshal/Interface-10                  193       6178584 ns/op     314.06 MB/s     1996611 B/op          2 allocs/op
BenchmarkTestdata/GolangSource/Encode/Token/Streaming-10             182       6550348 ns/op     296.24 MB/s       17776 B/op        168 allocs/op
BenchmarkTestdata/GolangSource/Encode/Token/Buffered-10              184       6475840 ns/op     299.65 MB/s        9344 B/op        153 allocs/op
BenchmarkTestdata/GolangSource/Encode/Value/Streaming-10             278       4267457 ns/op     454.71 MB/s    10599377 B/op        189 allocs/op
BenchmarkTestdata/GolangSource/Encode/Value/Buffered-10              333       3589382 ns/op     540.61 MB/s        8408 B/op        153 allocs/op
BenchmarkTestdata/StringEscaped/Marshal/Concrete-10                49074         24113 ns/op    1744.39 MB/s       18436 B/op          1 allocs/op
BenchmarkTestdata/StringEscaped/Marshal/Interface-10               40398         29844 ns/op    1409.42 MB/s       18449 B/op          2 allocs/op
BenchmarkTestdata/StringEscaped/Encode/Token/Streaming-10          37183         32241 ns/op    1304.59 MB/s       12624 B/op         29 allocs/op
BenchmarkTestdata/StringEscaped/Encode/Token/Buffered-10           38955         30725 ns/op    1368.97 MB/s        4080 B/op         19 allocs/op
BenchmarkTestdata/StringEscaped/Encode/Value/Streaming-10           4106        289498 ns/op     145.29 MB/s      151016 B/op        216 allocs/op
BenchmarkTestdata/StringEscaped/Encode/Value/Buffered-10            4256        281495 ns/op     149.42 MB/s       67632 B/op        201 allocs/op
BenchmarkTestdata/StringUnicode/Marshal/Concrete-10                49807         24134 ns/op     750.96 MB/s       18436 B/op          1 allocs/op
BenchmarkTestdata/StringUnicode/Marshal/Interface-10               40480         29812 ns/op     607.95 MB/s       18449 B/op          2 allocs/op
BenchmarkTestdata/StringUnicode/Encode/Token/Streaming-10          35911         32421 ns/op     559.02 MB/s       12624 B/op         29 allocs/op
BenchmarkTestdata/StringUnicode/Encode/Token/Buffered-10           38827         30860 ns/op     587.30 MB/s        4080 B/op         19 allocs/op
BenchmarkTestdata/StringUnicode/Encode/Value/Streaming-10          32996         36130 ns/op     501.63 MB/s       87456 B/op         33 allocs/op
BenchmarkTestdata/StringUnicode/Encode/Value/Buffered-10           41947         28643 ns/op     632.75 MB/s        4072 B/op         18 allocs/op
BenchmarkTestdata/SyntheaFhir/Marshal/Concrete-10                    174       6829914 ns/op     294.07 MB/s     3141771 B/op       2569 allocs/op
BenchmarkTestdata/SyntheaFhir/Marshal/Interface-10                   404       2942877 ns/op     682.49 MB/s     1146976 B/op          2 allocs/op
BenchmarkTestdata/SyntheaFhir/Encode/Token/Streaming-10              465       2569125 ns/op     781.78 MB/s       11832 B/op         77 allocs/op
BenchmarkTestdata/SyntheaFhir/Encode/Token/Buffered-10               474       2523400 ns/op     795.95 MB/s        3528 B/op         63 allocs/op
BenchmarkTestdata/SyntheaFhir/Encode/Value/Streaming-10              433       2758368 ns/op     728.15 MB/s     6646266 B/op         95 allocs/op
BenchmarkTestdata/SyntheaFhir/Encode/Value/Buffered-10               537       2225622 ns/op     902.44 MB/s        3840 B/op         61 allocs/op
BenchmarkTestdata/TwitterStatus/Marshal/Concrete-10                 1549        759064 ns/op     831.96 MB/s      501913 B/op         59 allocs/op
BenchmarkTestdata/TwitterStatus/Marshal/Interface-10                1165       1017282 ns/op     620.79 MB/s      469194 B/op          2 allocs/op
BenchmarkTestdata/TwitterStatus/Encode/Token/Streaming-10           1071       1115264 ns/op     566.25 MB/s       17464 B/op         94 allocs/op
BenchmarkTestdata/TwitterStatus/Encode/Token/Buffered-10            1094       1092146 ns/op     578.23 MB/s        9808 B/op         81 allocs/op
BenchmarkTestdata/TwitterStatus/Encode/Value/Streaming-10           1006       1167778 ns/op     540.78 MB/s     2551921 B/op        119 allocs/op
BenchmarkTestdata/TwitterStatus/Encode/Value/Buffered-10            1222        981393 ns/op     643.49 MB/s       13112 B/op         89 allocs/op
veqryn commented 10 months ago

looks like most of the benchmarks are within a half-percent of each other, then there are a few that are 2% faster with the PR, and a few more than are 3-4% slower with the PR. Margin of error is probably 2-3% on my laptop for small runs like this. How are things looking for you now?

veqryn commented 10 months ago

@dsnet The PR was updated with your suggestions. Please let me know what you think.

dsnet commented 10 months ago

Thanks for working on this, I'm a bit busy this week, so I might be able to get to this on the weekend.

veqryn commented 8 months ago

hey @dsnet , just wondering if we can get this merged in now :)

veqryn commented 6 months ago

rebased on master...

veqryn commented 6 months ago

hi @dsnet , I am now using JSON v2 in production, via my v2 json slog handler: https://github.com/veqryn/slog-json

However, because go.mod replacement rules don't work well for libraries, I am forced to point slog-json at my fork of JSON v2 in order to get the single-line pretty-printing.

When we left off with this PR it sounded like you were ready to merge it after a final re-review. Is there anything I can do to get this moving again, please?

dsnet commented 5 months ago

Generally, LGTM. There are some cleanups and minor behavior changes, but it'll probably easier to just change that after submission than have a few more rounds of back-and-forth review.

veqryn commented 5 months ago

Sounds great

dsnet commented 5 months ago

Thank you for the contribution. I apologize for the massive delay.

veqryn commented 5 months ago

You are very welcome