JSON-LD: streaming or canonicalization?

[VladimirAlexiev]

@Sveino We need to decide whether we want our JSON-LD to have one of these two desirable features:

• Streaming https://www.w3.org/TR/json-ld11-streaming/
  • Ordering the context first, and @id first in each block, enables streaming parsing. This means that only limited memory is needed to parse even huge models because triples can be emitted as soon as they are recognized, rather than having to be accumulated in memory. Speed is also increased because memory allocation is a slow process.
  • Asking for alphabetical order of nodes by @id precludes streaming generation because the generator will need to materialize the whole dataset (or at least all node URNs) in memory before outputting them
• Canonicalization (c14n) https://w3c-ccg.github.io/rdf-dataset-canonicalization/spec/
  • This is useful for comparing two datasets/models as strings, and enabling the crypto signing of CIM data
  • However, it is an expensive process and precludes streaming

I vote for streaming because CIM Differential models express the delta between two models explicitly. c14n does not help in computing the delta of two models, and is not needed for comparison: the delta expresses richer information than "two models are different".

[ericstephan]

Great point about the benefits of streaming versus canonicalization. The priority has always seemed to be integrity and change detection. Are you aware of use cases where performance is an issue in the energy sector?

[VladimirAlexiev]

AFAIK, Nordic44 comes to 850Mb RDF/XML. I guess the JSON-LD can be of a similar size. To answer this question, we need to define what is acceptable performance. Eg is 10s parsing time acceptable or not?