Empty values

[zeginis]

In some cases the data are sparce and there are many empty values. We can follow two approaches:

• Use null values -> "table": {"1999":[2311, null, null, null, null, null, null, null, 1522]}
• Use the position of the value -> "table": {"1999":["0": 2311, "8": 1522]}

For the record, json-stat adopts both approaches

[RickMoynihan]

I'd prefer to use one approach everywhere, rather than allow two different representations, which basically makes interop and all implementations harder.

I think the trade off between the two styles is to allow you to optimise for the case where you have more holes vs data, or more data vs holes.

I'd assumed the null, null, null... approach is what we'd use; i.e. that we'd assume we'd typically have more data than holes.

I think given how the API will guide users into slices that always have some data that sparseness is less of a problem, also we're not representing higher dimensional slices as json-stat does where sparseness is more an issue.

Additionally one of the primary motivations for this format is to request small subsets of the data for display; rather than send a whole dataset. Therefore we're deliberately not optimising the format for space efficiency; as it should almost always be relatively small - also regarding space usage I'd expect it to normally be gziped between the client/server anyway.

Let me know if you're happy with this or not.

[zeginis]

I agree that the null, null, null... approach is appropriate for our case.

[RickMoynihan]

Note there is a proposed exception to this in the PR #24, where if a whole row consists of nulls it will be dropped from the pages results; collapsing areas of total sparseness. This does also have some implications for the row headers dimension-values.