fxamacker / cbor

CBOR codec (RFC 8949) with CBOR tags, Go struct tags (toarray, keyasint, omitempty), float64/32/16, big.Int, and fuzz tested billions of execs.
MIT License
737 stars 60 forks source link

Add comparisons for fxamacker/cbor, encoding/gob, encoding/json #391

Closed x448 closed 1 year ago

x448 commented 1 year ago

Benchmark comparisons to Go standard library codecs are missing.

Let's add benchmark comparisons of fxamacker/cbor, encoding/gob, and encoding/json.

Encoded size comparisons are important for some use cases. Use simple table if there isn't enough time for charts. Make it readable on phone (I wish I had this available on my phone yesterday).

Use a common data struct with realistic values for comparison.

UPDATE: something like this...

Codec Benchmark Duration
ns/op  
Memory
B/op   
Allocs   
allocs/op
Encoded
Output 
fxamacker/cbor 2.5 Marshal (K,P) 201.4 176 2 80 bytes
fxamacker/cbor 2.5 Marshal (K) 421.7 176 2 80 bytes
fxamacker/cbor 2.5 Encode (K) 576.5 272 4 80 bytes
ugorji/go 1.2.11 Encode (K) 1,408.0 1320 4 80 bytes
encoding/json Encode (K) 850.9 368 4 129 bytes
encoding/json Encode 670.5 336 4 179 bytes
encoding/gob Encode 3,846.0 1,469 32 195 bytes
x448 commented 1 year ago

This will show toarray and keyasint features shrinking encoded data size.

I can simulate the keyasint feature for encoding/json. Didn't see any struct tags for encoding/gob.

fxamacker commented 1 year ago

@x448 This is a great idea. However, I'm closing this (for now) so we can revisit and maybe reopen after v2.5.0 is released.

Comparisons in README can require time to update when any codec compared is updated.

Lack of time to update the README can be the only blocker to a new release being tagged, so we should reduce size of README first by removing or updating outdated comparisons that already exist.