mverleg / binary_json

Binary encoding of JSON that emphasizes compression
Other
1 stars 0 forks source link

Consider whether object interning is worth it #2

Open mverleg opened 6 years ago

mverleg commented 6 years ago

Object interning would mean that for each object (map) with the same keys, the structure would be remembered. This would mean that for each occurrence, only the values need to be saved, while the keys and structure are only saved once.

This would be a great idea if we didn't also have string interning. Due to string interning, the keys of repeated objects take very little space even without interning. So perhaps this is not worth the cost in read-write speed.

mverleg commented 6 years ago

Another question would be whether the data type of the values is part of the interning. This could save more space for identical objects, but lead to more misses for objects with the same keys but different type values. I currently think it's probable for values to have the same types if keys are the same.

Some trick to handle null values may be needed in this case, since most languages have null as value for other types, rather than a type by itself (since JSON can't see if a null is supposed to be an int or a string).