search

ugorji / go

idiomatic codec and rpc lib for msgpack, cbor, json, etc. msgpack.org[Go]
MIT License
1.87k stars 295 forks source link

Poor performances with Json #64

Closed DAddYE closed 9 years ago

DAddYE commented 9 years ago

I made a quick test, source here: https://github.com/DAddYE/json-test

I'm pretty sure somewhere there is an error (maybe in my code?) but the speed of codec-json is very slow compared to the one in the standard library.

BenchmarkCodec         1    2461422589 ns/op     6400256 B/op     400017 allocs/op
BenchmarkStdLib 2000000000           0.07 ns/op        0 B/op          0 allocs/op

Installed codec with:

$ go get -tags=unsafe -u github.com/ugorji/go/codec/...
DAddYE commented 9 years ago

I think I found the issue. I had to use the bufio

See: https://github.com/DAddYE/json-test/commit/7a85b6de2dbc4e893e785c2260abb451fd4d979e

ugorji commented 9 years ago

Exactly. encoding/json internally creates a buffered reader internally. The problem is that it will end up reading more than is necessary for the json structure from the reader. It makes it hard to have a stream of multiple structures i.e. a multi-content stream.

codec instead expects that the user will handle buffering themselves, so will never read more than is necessary for its decoding. It thus allows you have a stream that contains a header, then multiple content streams with possible different encoding formats in sequence e.g.

etc.

DAddYE commented 9 years ago

It totally makes sense to me and I appreciated that