Ideas on reducing `meta`'s footprint

Beware! This issue is not meant to say "We need this now" or even "We need this" at all. Keeping SuperJSON human-readable is a big advantage that should not be underestimated. This issue just aims to keep ideas on how to compress it in case there's a need for that in the future, which most likely won't occur.

Idea 1: Nesting Meta

At the moment, our meta looks like the following:

{
  "a.deep.nested.path.containing.multiple.keys": "map",
  "a.deep.nested.path.containing.multiple.keys.a": "number",
  "a.deep.nested.path.containing.multiple.keys.b": "number",
}

(177 chars)

This is wasteful. It would be much better to have this nested like the following:

{
  "a": { "deep": { "nested": { "path": { "containing": { "multiple": { "keys": [
     "map",
     { "a": "number", "b": "number" }
  ] } } } } } }
}

(134 chars)

Or even better:

{
  "a.deep.nested.path.containing.multiple.keys": [
    "map",
    {
      "a": "number",
      "b": "number",
    }
  ]
}

(115 chars)

Idea 2: Hashing keys

Instead of storing the full key, we could compute a hash of it and store the prefix that uniquely identifies it amongst the other keys.

Example: Key: a.deep.nested.path Hash: 1108f7... (this is MD5, but any hash function works) Hashes of the other keys: 90fa1..., 994da..., 110d3....

These hashes can then be shortened to the following, making them uniquely identifiable: 1, 9, 99 and 11. They can then be matched to their original keys during revival by hashing the keys in value.

Beware: For undefined values, there are keys in meta that aren't present in value. They need to be accounted for.

Idea 3: Shortening meta descriptors

At the moment, the descriptors in meta are rather long. We could potentially enumerate them using integers or characters instead of lengthy strings.

Idea 4: Use non-JSON-encoding

meta does not necessarily need to be parsable for users outside of SuperJSON. To save a lot of characters, we could use some binary encoding or something like Avro or ProtoBuf.

flightcontrolhq / superjson