honeycombio / husky

a library for translating other data types to Honeycomb-format data structures
Apache License 2.0
2 stars 10 forks source link

Should we use jsoniter instead of encoding/json? #147

Closed cartermp closed 1 year ago

cartermp commented 1 year ago

From @ianwilkes in a recent chat:

but also we should use jsoniter, not json

Orthogonally to #145 and #146 we may want to look into using https://github.com/json-iterator/go. It looks like it can be a very simple drop-in replacement.

jsoniter purports to be over 150% faster at encoding in their benchmark, and json encoding is on our hot path for ingest. Does that track, or is it just some number on a table?

We could also use shepherd profiles in pyroscope to get a better sense of the data.

ianwilkes commented 1 year ago

We already use jsoniter in many places in hound for efficiency reasons - for example, for encoding query results in retriever client. The time and memory savings are real.

pkanal commented 1 year ago

Closed by #145 & #146 being merged