Introduce concept of "sealed” contexts

[gkellogg]

A number of standards track specifications use JSON-LD for extensibility but intentionally place limitations on overriding the terms defined in a core context. For example:

Implementations may augment the provided @context with additional @context definitions but must not override or change the normative context.

https://www.w3.org/TR/activitystreams-core/#jsonld

Implementations MUST produce an error when an extension JSON-LD Context overrides the expanded URL for a term specified in the base JSON-LD Context (https://w3id.org/credentials/v1). To avoid the possibility of accidentally overriding terms, developers are urged to scope their extensions.

https://w3c.github.io/vc-data-model/#extensibility

It seems like it would be a nice feature to allow enforcement of this desire by JSON-LD processors. Then authors could be assured that the interpretation of a context would be proper and properly implemented JSON-LD processors would throw errors if rules were violated.

In short, I propose we add a keyword with boolean value such as "@sealed": true (that could appear in contexts) to JSON-LD 1.1 that enables processors to enforce a desire to prevent defined terms from being redefined in subsequent contexts, but allows for new terms (aka extensions) to be defined.

Original issue: Introduce concept of "sealed" contexts #656

[gkellogg]

See #129.

[cwebber]

A couple of things:

• I like the wording in https://github.com/w3c/json-ld-syntax/commit/bd6f8aeff9770d7710ba2668bbafff2c66602f1d
• I talked with @dlongley about whether or not there are any terms that are less "overloaded" than "sealed". In general what we're exploring is referred to "shadowing" in the context of scoping in programming languages. As such, we think that instead of @sealed: true it might be much better to have @allowShadow: false. What do people think?

It took me a long time to read through this and understand the conversation, but I think where it ended at is good. I think we shouldn't make things more complicated than the current change; we should be able to explain sealing/restricting shadowing as a single rule that potentially composes with another rule, and I think the current language does that nicely.

We should be clear though that sealing or whatever doesn't help json-only applications be "secure" in that they know for sure that a term is going to be the term they think it is; someone could always be intentionally tricky through context nullification.

[cwebber]

Now to talk about implementation. Here's how I think it can be done:

• Add a new slot/property to the active context that is something like sealedTerms or unshadowable. This is a set of terms that can't be shadowed, and it's append-only.
• At the time of adding a new term with @allowShadow: false (or whatever) we add it to that set.
• Further attempts to define a term in that set should throw an error by default. It's possible that we can configure things to warn instead, but I feel pretty strongly that an error is better behavior, otherwise we might have users unintentionally produce a different document than they thought they did.
• Nullification simplify resets the entire active context to a "fresh" one, including the sealed terms.

[dlongley]

We should be clear though that sealing or whatever doesn't help json-only applications be "secure" in that they know for sure that a term is going to be the term they think it is; someone could always be intentionally tricky through context nullification.

I disagree. These rules are supposed to make it such that if you follow a specification you should be safe. If you go off into extension land with no spec that makes it clear what checks you have to run, then you aren't. We'll need to make sure that context nullification of sealed terms is only possible through the creation of a term with a nested scope (no type-based scope context nullification or embedded context nullification of existing sealed terms).

[dlongley]

JSON developers understand "isolated extensions" to mean "a new term I define that can be used to create a new branch in the JSON tree underwhich all of the term definitions are wiped clean". This is the only place where context nullification that clears all sealed terms should be permitted. This means embedded context nullification is not permitted and terms with scoped contexts that appear in the type position cannot nullify sealed terms. Only a new term definition (with a scoped null context) where that term appears in a non-type position can nullify sealed terms -- as this matches the expectations of the core constituency of this feature.

[gkellogg]

@cwebber said:

Now to talk about implementation.

Note the implementation is described in w3c/json-ld-api#60, which I've implemented myself.

[gkellogg]

@dlongley said:

We'll need to make sure that context nullification of sealed terms is only possible through the creation of a term with a nested scope (no type-based scope context nullification or embedded context nullification of existing sealed terms).

We discussed this at the F2F and felt that it was too onerous and complicated things. Authors are creating these documents and choosing to add the sealed contexts in the first place, which they could subvert in other ways. Trying to make this too bullet-proof is ultimately impossible, and the nuclear option of nulling out the context is sufficiently rarely used now that it's not worth overly protecting. I could see generating a warning, if this is done with sealed terms in scope. Jump onto the next call to get a deeper discussion, but we spent quite a bit of time discussing this last Friday.

[dlongley]

@gkellogg,

We discussed this at the F2F and felt that it was too onerous and complicated things... Trying to make this too bullet-proof is ultimately impossible...

I respectfully disagree. Without the above limitation the feature doesn't fully solve the use case. With it, I believe it does. I also believe it matches expectations for how one extends JSON. If we're able to make it so, the feature should provide the guarantees that the main users of the feature expect.

Authors are creating these documents and choosing to add the sealed contexts in the first place, which they could subvert in other ways.

I'm not sure what you mean by this. The target use case is not potentially random combinations of contexts attached to documents floating about the Web. I agree that there should be no expectation for what you can trust there. Use a JSON-LD processor to mitigate your risk.

But we absolutely should be able to create W3C specifications that truthfully claim application developers can use JSON-LD without doing JSON-LD processing if they run a couple of simple checks. Anything short of that lends credence to the argument that you can't really use JSON-LD without running a JSON-LD processor.

Jump onto the next call to get a deeper discussion, but we spent quite a bit of time discussing this last Friday.

Ok.

[azaroth42]

I fully support the consensus that if the feature must be watertight, then it is impossible without breaking compatibility with 1.0. 1.0 specifies that a null context wipes out all definitions, or setting a term to null wipes out that definition. See Section 6.7. We can't change that.

FWIW, this topic is not on the agenda for the call on 2019-02-15.

If there is time to revisit this before CR, then we will. If not, then we can take the feature out completely or leave it as we have specified. But we are not, in the short term, going to go around the circle yet again and especially not without stakeholders being present and engaged with the work of the group.

[azaroth42]

But we absolutely should be able to create W3C specifications that truthfully claim application developers can use JSON-LD without doing JSON-LD processing if they run a couple of simple checks.

And the check is that no context uses null to wipe out a term or the active context.

[dlongley]

@azaroth42,

I fully support the consensus that if the feature must be watertight, then it is impossible without breaking compatibility with 1.0. 1.0 specifies that a null context wipes out all definitions, or setting a term to null wipes out that definition. See Section 6.7. We can't change that.

I don't think this position makes any sense. We can change 1.0 behavior regarding how select terms can be overridden but we can't change 1.0 behavior regarding how all such terms can be overridden at once? That reasoning sounds like it relies upon an inconsistent understanding of backwards compatibility.

Rather, no terms were @sealed in JSON-LD 1.0 because the feature did not exist. We may freely define in 1.1 how these new special terms are created and when they are overridden, cleared, or not.

These new terms should behave in the fashion that best supports their use case and best fulfills the expectations of their main users. I think this means we should make it such that they are only cleared as in the example that's in the spec today. Namely, that you must define a new term that explicitly clears them through a scoped null context and the clearing only applies when that term is used to extend the JSON tree in the data (as opposed to just augmenting data with a type). This makes the feature work in the best way for the users of the feature and relieves the most frustration for those exposed to JSON-LD who do not want to use a JSON-LD processor.

Most importantly, it best helps deliver on a promise the community has been making since JSON-LD was created: You can easily use JSON-LD as regular JSON without having to process it.

And the check is that no context uses null to wipe out a term or the active context.

This check still requires JSON-LD processing which harms the purpose of the feature. It adds complexity on the users of the feature that is not necessary if we limit clearing @sealed terms to requiring the definition of a new term with a scoped null context that is used to extend the JSON tree. I believe the feature will behave as expected (to the users of the feature) if we do this.

I haven't seen a good argument against this definition for clearing @sealed terms yet. As argued, I believe the above backwards compatibility argument is flawed.

[iherman]

This issue was discussed in a meeting.

• No actions or resolutions
  View the transcript

  Sealed Contexts’ timing
  Rob Sanderson: a discussion started yesterday about the decisions made during F2F about sealed contexts
  … we need to see to which extent the current proposal works and does not work with existing use cases;
  … we have spent several week and close to half the F2F time to talk about it;
  … continuing is not in the interest of the group.
  Dave Longley: We plan on implementing next week and we’ll engage on Github
  … we are globally happy with the decision of the F2F,
  … but we need sealing to be a little more restrictive.
  … We will come up with an implementation of what we need.
  Rob Sanderson: this will help to assess how much additional complexity is required
  Gregg Kellogg: what we have been looking at was doable (I did implement it),
  … but we still decided that it added too much complexity.
  … That being said, I’m happy to discuss with you about it.
  Dave Longley: All we want to do is add constraints on when the context can be cleared with ‘null’.
  … We don’t think it adds that much complexity.
  

[azaroth42]

Resolved :)