w3c / json-ld-syntax

JSON-LD 1.1 Specification
https://w3c.github.io/json-ld-syntax/
Other
112 stars 22 forks source link

Is there a scope context mechanism for clearing context definitions? #98

Closed dlongley closed 5 years ago

dlongley commented 5 years ago

Is there a way to use the scoped context mechanism to "embed" information in a JSON-LD document such that all term definitions are cleared? So, for example, suppose we had this:

{
  "@context": {
    "@version": 1.1,
    "someTermToClear": "ex:someTermToClear",
    "record": {"@id": "ex:record", "@container": "@graph", "@context": null}
  },
  "someTermToClear": "should be defined here",
  "record": {
    "@context": {
      "someOtherStuff": "ex:someOtherStuff"
    },
    "@id": "ex:someId",
    "someOtherStuff": "this should be defined here",
    "someTermToClear": "this should NOT be defined here"
  }
}

The goal here is to avoid requiring the author of record value (or a user of it) to have to know to clear any terms in an "outer container". This may allow "safer" composition/embedding of JSON-LD documents.

dlongley commented 5 years ago

By the way, the exact example given above does not work in the playground, the term someTermToClear is still defined in the embedded record.

ajs6f commented 5 years ago

I'm probably missing the point, but can you not unset someTermToClear in the inner context? The author of the data there needn't know about that, just the context author...

dlongley commented 5 years ago

@ajs6f,

The @context author doesn't know anything about the "enveloping" structure. The point is to be able to reuse contexts when embedding data -- without breaking abstractions (having to rewrite/change those contexts such that JSON-LD processing becomes required).

ajs6f commented 5 years ago

@dlongley Ok, makes sense. How would this play with "sealed" contexts?

dlongley commented 5 years ago

@ajs6f,

Yes, this is related to "sealed" contexts. I think one of the properties of sealed contexts should be to enable term overrides using embedded contexts. It may be that an embedded @context can simply override sealed terms (because it's explicitly in the data) but it may be better if this is only permitted if the sealed context explicitly uses a scoped term that clears term definitions in order to enable it. Does that make sense?

ajs6f commented 5 years ago

The first part does, IIUC: contexts in the instance would override contexts from elsewhere, even if they don't "want" to be overridden, because the instance author is the ultimate authority. I'm not quite sure I follow the second point. "a scoped term that clears term definitions"-- that means a new keyword that would drop the definitions? That keyword would appear in the (outer) sealed context?

dlongley commented 5 years ago

For "sealed" contexts, suppose a context array like this:

[<sealed context 1>, <sealed context 2>, <maybe sealed context 3>]

In this case, a JSON-LD processor should ignore any "overriding" term definitions provided in context 2 and 3, always keeping the definition from context 1. This much is clear. But what happens in the case of an embedded context?

{
  "@context": "[<sealed context 1>, <sealed context 2>, <maybe sealed context 3>]",
  "record": {
    "@context": {
      "i_am_embedded": "ex:foo"
    }
  }
}

Here, I would expect us to need additional/different behavior. And it may turn out that it would be nice to have this different behavior whether a context is sealed or not. Essentially, being able to say "treat this document as an embedded one that knows nothing about its parent information" is valuable. It may be, that for sealed contexts, we will need to "enable" this to happen ... or it could happen automatically, but that seems more dangerous.

dlongley commented 5 years ago

@ajs6f,

I can imagine core W3C data model specs defining some kind of JSON tree that looks like this:

{
  "@context": "some_sealed_context",
  "service": {
    "id": "ex:fooService",
    "endpoint": "https://example.com"
  }
}

Now, if an instance author does this:

{
  "@context": "some_sealed_context",
  "service": {
    "@context": {
      "endpoint": "ex:haha_tricked_you",
      "realEndpoint": "ex:endpoint"
    },
    "id": "ex:fooService",
    "endpoint": "https://junk.example.com",
    "realEndpoint": "https://example.com"
  }
}

All the software that performs JSON-LD processing will use a different endpoint from the software that does not. I don't think we can assume that an embedded context can always override terms. I think we may want to be able to say "here's where you can override terms/here's where terms must be overridden". It's a more clean way of calling out where extensions are expected to be or where "embedded documents" can more freely override (anything).

dlongley commented 5 years ago

@ajs6f,

I'm not quite sure I follow the second point. "a scoped term that clears term definitions"-- that means a new keyword that would drop the definitions? That keyword would appear in the (outer) sealed context?

Yes, it would look something like this:

{
  "@context": {
    "extensionsGoHere": {"@id": "ex:extension", "@context": null}
  }
  "extensionsGoHere": {
    "@context": "whatever"
  }
}
ajs6f commented 5 years ago

It seems like we might be talking about a few things here:

  1. the relationship between sealed and unsealed contexts,
  2. the relationship between retrieved and instance-embedded contexts,
  3. and the relationship between inner and outer contexts, whatever their source.

So there's a lot of ways those things could interact (I kind of want to write down a combinatorial table here, but I'm not sure what I would be computing!)

I'd like to try and simplify the discussion (just for a bit). Perhaps we can leave aside sealing (I realize that we can only do that for a little while) and think about unsealed contexts only and the latter two relationships. In a world without sealed contexts (just for now! :) ) what would this feature look like? Would it enable instance contexts to easily blank out the computed context and "start afresh"? I'm just trying to get my head around a small piece of this before we reintroduce sealing, because sealing is also hypothetical for the moment and I'd like to minimize the hypotheticals in my brain. It's not much larger than Pooh's. :dizzy_face:

dlongley commented 5 years ago

@ajs6f,

I agree that we're talking about a number of different things and how they interact. :)

Perhaps we can leave aside sealing (I realize that we can only do that for a little while) and think about unsealed contexts only and the latter two relationships. In a world without sealed contexts (just for now! :) ) what would this feature look like? Would it enable instance contexts to easily blank out the computed context and "start afresh"?

Yes, it would enable instance contexts to blank out the computed context and "start afresh". In particular, it would do so for certain terms (i.e. any objects under the term "x" would have a "fresh start"). The use case for this would be to ensure that nothing "leaks in" from the original instance context into whatever is embedded within a certain term, without undefining the term itself. It enables encapsulation and, because of that, I would expect it to often be used in conjunction with "@container": "@graph".

gkellogg commented 5 years ago

Setting aside the sealed/frozen nature, setting either @context or a given term to null clears it out. What should happen for frozen contexts or terms defined in a frozen context?

Arguably, setting @context to null should clear out any frozen contexts, as it's not modifying them and is a rather specific directive.

How to clear a sealed/frozen term definition? This needs discussion, if we want to allow it at all.

dlongley commented 5 years ago

@gkellogg,

setting either @context or a given term to null clears it out.

The example given in the OP does not clear out all the terms on the playground. Are you suggesting that it should and it's a problem with the playground (and that this is already spec'd)?

gkellogg commented 5 years ago

@ajs6f said:

I think one of the properties of sealed contexts should be to enable term overrides using embedded contexts.

Sounds like adding a fair bit of complication. Is this a real-world concern?

gkellogg commented 5 years ago

Step 3.1 of the Context Processing Algorithm indicates that null starts everything afresh:

If context is null, set result to a newly-initialized active context and continue with the next context.

dlongley commented 5 years ago

@gkellogg,

Step 3.1 of the Context Processing Algorithm indicates that null starts everything afresh:

Yes, that's true, but what of scoped contexts?

@ajs6f said:

I think one of the properties of sealed contexts should be to enable term overrides using embedded contexts.

Sounds like adding a fair bit of complication. Is this a real-world concern?

I have a real world use case of embedding sub documents (which are treated as sub graphs) that should not have to concern themselves at all with the sealed context of the container:

{
  "@context": "some outer sealed stuff",
  "type": "SomeOperationToAddARecordToABlockchain",
  ...,
  "record": {
    "@context": "some inner stuff that doesn't care/know about outer stuff",
    ...
  }
}
gkellogg commented 5 years ago

Yes, that's true, but what of scoped contexts?

The processing should be the same, as it's treated as if the "@context": null appears inline. Certainly worth a test case.

I have a real world use case of embedding sub documents (which are treated as sub graphs) that should not have to concern themselves at all with the sealed context of the container

Perhaps it should do the following:

{
  "@context": "some outer sealed stuff",
  "type": "SomeOperationToAddARecordToABlockchain",
  ...,
  "record": {
    "@context": [null, "some inner stuff that doesn't care/know about outer stuff"],
    ...
  }
}
ajs6f commented 5 years ago

@ajs6f said: I think one of the properties of sealed contexts should be to enable term overrides using embedded contexts.

Just for the record, I did not say this, @dlongley did.

ajs6f commented 5 years ago

@dlongley I think part of the difficulty for me here is that the gesture of blanking out a term or the whole context is something sealing is supposed to disallow, right?

So we've got at least three categories: the original sealed context, contexts that are not allowed to alter it, and contexts that can. Is it possible to partition the last two on a scope boundary, or on the boundary between retrieved and embedded contexts? Or must we think about a new and fresh way to signify a "able to defeat sealing" context? (Also, we might want a name for that category, if only for discussion.)

dlongley commented 5 years ago

@gkellogg,

Perhaps it should do the following:

See below -- I think this needs to be done from within the sealed context.

@ajs6f,

I think part of the difficulty for me here is that the gesture of blanking out a term or the whole context is something sealing is supposed to disallow, right?

Yes. I think it should only be permitted if the sealing context allows for it by clearing the context.

So we've got at least three categories: the original sealed context, contexts that are not allowed to alter it, and contexts that can. Is it possible to partition the last two on a scope boundary, or on the boundary between retrieved and embedded contexts? Or must we think about a new and fresh way to signify a "able to defeat sealing" context? (Also, we might want a name for that category, if only for discussion.)

I think the sealing context has to enable "defeat" in targeted areas by clearing the context for those terms via scoped contexts (or using some kind of @container, but I think simply clearing the context in a scoped way would be better).

iherman commented 5 years ago

I may not understand the whole issue... but it looks to me as if we wanted to have some @sealed flag with a boolean value, and which can be used for the context as a whole as well as for a specific terms: This means that:

"@context" "term" Can "term" be redefined?
"@sealed": true "@sealed": true no
"@sealed": true "@sealed": false yes
"@sealed": false "@sealed": true no
"@sealed": false "@sealed": false yes

Obviously, there are defaults: if the context is sealed then, by default, all terms are sealed and if context is not sealed then, by default, non of the terms are sealed.

Can this solve the use cases, @dlongley?

dlongley commented 5 years ago

@iherman,

Can this solve the use cases, @dlongley?

I think the table is good and makes sense and would solve some use cases. I think it would also at least partially solve the use case here, but perhaps not entirely. Let me work through it below.

So, a use case that we have is that we want a sealed context to be able to say that the term foo can be redefined, but only for its own scope (i.e. its own values and any more deeply nested in its tree). More specifically, we'd like to wipe out all term definitions within the scope of foo, but let's stick with the redefinition first.

So, this would be insufficient:

{
  "@context": [
    {
      "@sealed": true,
      "bar": "ex:bar",
      "foo": {"@id": "ex:foo", "@sealed": false}
    },
    {
      "foo": "ex:not_foo"
    }
  }],
  "foo": "this should be 'ex:foo' for this use case, but it is not"
}

Because it allows foo to be redefined globally (not scoped). Obviously, the above could be fine for some other use case. But here, to be clear, we only want foo to change like this:

{
  "@context": "...",
  "foo": { /* `foo` should always be 'ex:foo' here */
    "foo": "this can be 'ex:not_foo', no problem"
  }
}

So, you can see that we'd need something more than just saying "@sealed": false in the term definition in order to create the proper restriction. We'd need to scope it. So perhaps this could be achieved like this:

{
  "@context": [
    {
      "@sealed": true,
      "bar": "ex:bar",
      "foo": {
        "@id": "ex:foo",
        "@context": {
          "ex:foo": {"@id": "ex:foo", "@sealed": false}
        }
      }
    }],
  "foo": { /* yay, this would still be `ex:foo` */
    "@context": {
      "foo": "ex:not_foo"
    },
    "foo": "this can be 'ex:not_foo', no problem"
  }
}

But that could get quite verbose for many terms. Of course, this could be possibility avoided like this:

{
  "@context": [
    {
      "@sealed": true,
      "bar": "ex:bar",
      "foo": {
        "@id": "ex:foo",
        "@context": {"@sealed": false}
      }
    }],
  "foo": { /* yay, this is still 'ex:foo' */
    "@context": {
      "foo": "ex:not_foo"
    },
    "foo": "this can be 'ex:not_foo', no problem"
  }
}

This would not achieve the goals of this particular issue, which is to automatically clear all term definitions within the "scope" of "foo", but it would "unseal" all term definitions to allow for them to be redefined. I believe this also enables us to say that an "embedded" context does NOT automatically unseal anything, rather, the sealed context MUST explicitly use scoped contexts to say where any terms can be unsealed. So a big +1 to this for dealing with the sealing/unsealing issue on its own.

Now, regarding this particular issue, per @gkellogg's comment it seems to me that clearing all term definitions should actually work per the spec today, but does not in the implementations. So we may just have a bug. We may need further spec clarification (not sure?). But fixing the bug would ensure that this works too:

{
  "@context": [
    {
      "@sealed": true,
      "bar": "ex:bar",
      "foo": {"@id": "ex:foo", "@context": null}
    }
  ],
  "foo": { /* yay, this is "ex:foo" */
    "@context": {
      "foo": "ex:not_foo"
    },
    "foo": "this can be 'ex:not_foo', no problem"
  }
}

I think the above with "@context": null would need to unseal everything automatically ... unless we required this in order to do that:

{
  "@context": [
    {
      "@sealed": true,
      "bar": "ex:bar",
      "foo": {"@id": "ex:foo", "@context": [null, {"@sealed": false}]}
    }
  ],
  "foo": { /* yay, this is "ex:foo" */
    "@context": {
      "foo": "ex:not_foo"
    },
    "foo": "this can be 'ex:not_foo', no problem"
  }
}

What do you think?

I think if we can support some variant of the above we'll cover all the cases.

iherman commented 5 years ago

I think the above with "@context": null would need to unseal everything automatically ...

I am trying to find a clear way of explaining all this that does not sound to be too complicated. What I had in mind was:

  1. "@context":null wipes out the previous @context-s altogether, as if they were not there at all. Ie, yes, it would also unseal everything automatically.
  2. A term level "@sealed" value overwrites the global or "inherited" seal status for a term, where "inheritance" may be either due to a previous @context in an array, or a "parent" @context in case of an embedded one.

If I understand your examples, it does solve the various issues, albeit a bit verbose here and there. But a bit of verbosity is, I believe, acceptable if the processing rules are clear...

iherman commented 5 years ago

This issue was discussed in a meeting.

iherman commented 5 years ago

This issue was discussed in a meeting.

gkellogg commented 5 years ago

Other than for the potential for a scoped context within a sealed context which can unseal the context underneath the term, scoped contexts should always be thought of as a short-hand for simply defining the scoped context embedded inline. So for example, @dlongley's original example is equivalent to the following:

{
  "@context": {
    "@version": 1.1,
    "someTermToClear": "ex:someTermToClear",
    "record": {"@id": "ex:record", "@container": "@graph"}
  },
  "someTermToClear": "should be defined here",
  "record": {
    "@context": [null, {"someOtherStuff": "ex:someOtherStuff"}],
    "@id": "ex:someId",
    "someOtherStuff": "this should be defined here",
    "someTermToClear": "this should NOT be defined here"
  }
}

In the proposed change in https://github.com/w3c/json-ld-syntax/issues/20#issuecomment-455640995, only a scoped context definition within a sealed context/term can be used to unseal that context by setting it to null. You can't do the same thing outside of a term definition.

iherman commented 5 years ago

This issue was discussed in a meeting.