Closed dlongley closed 4 years ago
Note that the frame will need to have @graph
to start with the default graph, rather than the merged graph.
It should do more than that, although my implementation doesn't quite get there. Use the following frame in my distiller:
{
"@context": {
"@version": 1.1,
"id": "@id",
"claim": {"@id": "ex:claim"},
"name": "ex:name",
"proof": {"@id": "ex:proof", "@type": "@id"},
"signer": {"@id": "ex:signer", "@type": "@id"},
"subject": {"@id": "ex:subject", "@type": "@id"},
"knows": {"@id": "ex:knows", "@type": "@id"}
},
"@graph": {
"subject": {
"@embed": "@always"
},
"proof": {}
}
}
I get the following output:
{
"@context": {
"@version": 1.1,
"id": "@id",
"claim": {
"@id": "ex:claim"
},
"name": "ex:name",
"proof": {
"@id": "ex:proof",
"@type": "@id"
},
"signer": {
"@id": "ex:signer",
"@type": "@id"
},
"subject": {
"@id": "ex:subject",
"@type": "@id"
},
"knows": {
"@id": "ex:knows",
"@type": "@id"
}
},
"@graph": [
{
"ex:proof": {
"@graph": [
{
"@type": "ex:Proof",
"name": "the proof",
"signer": "ex:subject"
},
{
"id": "ex:subject",
"name": "something different"
}
]
},
"id": "ex:cred",
"subject": {
"id": "ex:subject",
"knows": {
"id": "ex:issuer",
"name": "Someone else"
},
"name": "the subject"
}
}
]
}
The JavaScript implementation doesn't quite do this, and generates an error in any case using this frame.
I'll need to see why the proof
term isn't being used, and why we're not dropping the @graph
elements. We also don't embed ex:subject
, which needs to be investigated.
The step at 4.4.1 in the Framing Algorithm should handle finding the graph and embed it. It should also work on the merged graph, I think, but doesn't seem to for me. I need to see if these are implementation issues, or if the algorithm is missing something. Help appreciated if you can look at it from your end.
This issue was discussed in a meeting.
RESOLVED: close framing#27 wontfix, as there’s no justification for the required RDF layer requirement that the blank node identity of the named graph is the default subject of the triples in the graph {: #resolution15 .resolution}
@iherman,
The conversation doesn't seem to apply to this issue but rather to this other one about SHeX and blank node identifier reuse:
https://github.com/w3c/json-ld-api/issues/26
Was the wrong issue closed?
This issue has to do with a bug in either the framing algorithm or the spec (or both) ... it is unrelated to SHeX or reusing blank nodes with named graph subjects/etc.
@dlongley I just did what the resolution said in the minutes... @azaroth42, has there been a mistake in the minutes?
(I am on my way to the airport, I cannot check it now.)
Yes, this is an issue we didn’t discuss and needs to be reopened.
Can somebody send me the changes on the minutes? Better: do the changes?
For the records: https://github.com/w3c/json-ld-framing/issues/27#issuecomment-462051772 is indeed unrelated to this issue. Change happened on the minutes on 2019-02-15.
This issue was discussed in a meeting.
RESOLVED: Leave framing 27 open, and to investigate the cause. Record open issue in the next PWD of Framing. {: #resolution2 .resolution}
So, I re-tried this with the following frame:
{
"@context": {
"@version": 1.1,
"id": "@id",
"claim": {"@id": "ex:claim"},
"name": "ex:name",
"proof": {"@id": "ex:proof", "@type": "@id", "@container": "@graph"},
"signer": {"@id": "ex:signer", "@type": "@id"},
"subject": {"@id": "ex:subject", "@type": "@id"},
"knows": {"@id": "ex:knows", "@type": "@id"}
},
"@graph": {
"subject": {
"@embed": "@always"
},
"proof": {}
}
}
I do get mostly the expected results:
{
"@context": {
"@version": 1.1,
"id": "@id",
"claim": {
"@id": "ex:claim"
},
"name": "ex:name",
"proof": {
"@id": "ex:proof",
"@type": "@id",
"@container": "@graph"
},
"signer": {
"@id": "ex:signer",
"@type": "@id"
},
"subject": {
"@id": "ex:subject",
"@type": "@id"
},
"knows": {
"@id": "ex:knows",
"@type": "@id"
}
},
"id": "ex:cred",
"subject": {
"id": "ex:subject",
"name": "the subject",
"knows": {
"id": "ex:issuer",
"name": "Someone else"
}
},
"proof": [
{
"@type": "ex:Proof",
"name": "the proof",
"signer": {
"id": "ex:subject",
"name": "something different"
}
},
"ex:subject"
]
}
The issuer
property is missing, but it's not defined in the context. By adding @container: @graph
, the proof
property is used, and contains the graph value, although it's in an array, and probably shouldn't be with out the @set
being present in the context definition.
The playground still doesn't generate the right output, however, but it seems that the algorithm is (mostly) correct.
Actually, the only oddity here is that "proof" contains two items, the second of which is "ex:subject", and I'm not sure where that comes from.
Actually, the only oddity here is that "proof" contains two items, the second of which is "ex:subject", and I'm not sure where that comes from.
This is because an added restriction is necessary to get a single item, which is done in #63.
@gkellogg,
This is because an added restriction is necessary to get a single item...
So while the output in #63 looks right to me, I don't understand why adding the type
should be required to avoid bringing in ex:subject
as another match for proof
. In fact, I still don't understand where that's coming from, it seems like an artifact that wouldn't generate any quads. Furthermore, the original data does not have a proof
relationship that points at two blank node named graphs, rather it points at just one... anyway, I'm confused about that.
This issue was discussed in a meeting.
@include?
Some observations:
@embed: @never
at the top level, which prevents anything but a node reference from being added.
@never
(or other embedding restrictions) if we are actually embedded, and not at the top level (may require some additional state).embedded
flag to the framing state, defaulting to false
, and set to true
when recursing to embedded nodes would seem to address this.@included
within the first/arbitrary object,@gkellogg,
- Seems like we should only honor the
@never
(or other embedding restrictions) if we are actually embedded, and not at the top level (may require some additional state).
Would this make it impossible to use @never
to produce output with only node references? For example:
Input:
{
"@graph": [{
"@id": "ex:foo1",
"@type": "ex:Foo"
}, {
"@id": "ex:foo2",
"@type": "ex:Foo"
}, {
"@id": "ex:bar",
"@type": "ex:Bar"
}]
}
Frame:
{
"@type": "ex:Foo",
"@embed": "@never"
}
Output:
{
"@graph": [
{
"@id": "ex:foo1"
},
{
"@id": "ex:foo2"
}
]
}
I don't know if that's a use case or not -- but I suppose it could be. I think that's my only potential concern with the approach.
- In compaction, I think more work for graph containers is necessary, and use @included if there is more than one.
- We could also put @included within the first/arbitrary object, or also not select that property if there’s more than a single top-level node object, which may make sense too.
So here are my thoughts on the two options:
My first thought was that seeing @included
in the output when using "@container": "@graph"
would be more undesirable/unexpected. Post framing, you'd have to check for this special case within your results. This suggested to me that we ought to treat it like a non-match. The whole point of using "@container": "@graph"
is that you're defining a term that is intended to have a value of one or more blank node named graphs and you're looking for a clean, compact way to express that. Injecting an @included
in there is at odds with that.
However, on second thought, I think it would be odd to make it impossible to compact to an "@container": "@graph"
term when you've got a blank node named graphs with multiple roots (i.e., you're using @included
). We just introduced this @included
term to make it easier for people to do things just like that. I imagine authors may eventually craft JSON-LD that does just this and they'd be quite surprised that it would not round trip using their context. It feels like preventing a term match here would therefore be creating a special case which we should avoid.
So, I think using @included
seems like the best path forward of the two options. I think frame authors should either focus on creating frames that ensure @included
won't surface or be ready to handle/reject results where it does.
Also, I didn't quite understand this comment (emphasis mine):
We could also put
@included
within the first/arbitrary object
I may be misunderstanding what you're saying here, but it wouldn't be arbitrary nor would I think it would affect compaction because @included
would be in the expanded output produced during framing.
So we'd get something like this from framing:
// top level
"@graph": [{
// will be come a `@graph: @container`
// post compaction
"ex:proof": [{
// first blank node named graph
"@graph": [{
// two matches at the graph root level
"@included": [{
"@id": "ex:foo1",
"@type": ["ex:Foo"]
}, {
"@id": "ex:foo2"
"@type": ["ex:Foo"]
}]
}]
}, {
// second blank node named graph
"@graph": [{
// just one match at the graph root level
"@id": "ex:bar1",
"@type": ["ex:Bar"]
}]
}]
}]
Compaction would just run normally on that and produce:
// top level
"@context": {
"proof": {"@id": "ex:proof", "@container": "@graph"}
},
"@graph": [{
"proof": [{
// first blank node named graph
// two matches at the root graph level
"@included": [{
"@id": "ex:foo1",
"@type": "ex:Foo"
}, {
"@id": "ex:foo2"
"@type": "ex:Foo"
}]
}, {
// second blank node named graph
// just one match at the root graph level
"@id": "ex:bar1",
"@type": "ex:Bar"
}]
}]
So, framing would handle injecting @included
based on the same named graph where multiple matches occurred. So, if you're inside a named graph and you're at the "root level" of that named graph, and we find more than one match, then we have to wrap the matches in an object with a key of @included
and a value that is the array of matches -- and use that as the single value for the named graph @graph
array. With just one match, we use it directly as the value for the named graph @graph
array. If this is what you meant then we're on the same page. But it seems to me like this wouldn't be a modification to compaction at all.
In fact, maybe we could get away with always outputting @included
for named graph matches and letting the compaction step of framing remove it when there's just one node (does compaction do that now?). I don't know what would be easier/more efficient, just putting that out there.
- It seems we should not add a node at the top level if it is embedded, but this creates a processing order dependency.
I thought @once
covered most of these cases already -- and that we didn't add something to the top level unless it matched the frame up there. If it matches at the top level and elsewhere, I don't see why we'd exclude it at the top level in particular. That seems like it would have to be a different @embed
mode -- and that it would introduce more complexity. Do we really need to do anything here?
@gkellogg,
- Seems like we should only honor the
@never
(or other embedding restrictions) if we are actually embedded, and not at the top level (may require some additional state).Would this make it impossible to use
@never
to produce output with only node references? For example:Input:
{ "@graph": [{ "@id": "ex:foo1", "@type": "ex:Foo" }, { "@id": "ex:foo2", "@type": "ex:Foo" }, { "@id": "ex:bar", "@type": "ex:Bar" }] }
Frame:
{ "@type": "ex:Foo", "@embed": "@never" }
Output:
{ "@graph": [ { "@id": "ex:foo1" }, { "@id": "ex:foo2" } ] }
I don't know if that's a use case or not -- but I suppose it could be. I think that's my only potential concern with the approach.
Yes, this wouldn't be supported, but that's not embedding. The @embed
flag is about embedding nodes within other nodes, not about expressing top-level node references, for which there really is no use case as they simply describe free flowing nodes with no incoming or outgoing vertices. I'd say it's an anti-pattern to use node references with no incoming links.
From the syntax doc:
A common idiom found in JSON usage is objects being specified as the value of other objects, called object embedding in JSON-LD...
Embedding is a JSON-LD feature that allows an author to use node objects as property values. This is a commonly used mechanism for creating a parent-child relationship between two nodes.
(As an aside, the algorithm should probably also display node references as immediate values of @included
.)
- In compaction, I think more work for graph containers is necessary, and use @included if there is more than one.
- We could also put @included within the first/arbitrary object, or also not select that property if there’s more than a single top-level node object, which may make sense too.
So here are my thoughts on the two options:
My first thought was that seeing
@included
in the output when using"@container": "@graph"
would be more undesirable/unexpected. Post framing, you'd have to check for this special case within your results. This suggested to me that we ought to treat it like a non-match. The whole point of using"@container": "@graph"
is that you're defining a term that is intended to have a value of one or more blank node named graphs and you're looking for a clean, compact way to express that. Injecting an@included
in there is at odds with that.
👍
However, on second thought, I think it would be odd to make it impossible to compact to an
"@container": "@graph"
term when you've got a blank node named graphs with multiple roots (i.e., you're using@included
). We just introduced this@included
term to make it easier for people to do things just like that. I imagine authors may eventually craft JSON-LD that does just this and they'd be quite surprised that it would not round trip using their context. It feels like preventing a term match here would therefore be creating a special case which we should avoid.
But we have many cases where you can't compact using a term which doesn't match. I think it's in perfect keeping with the spirit of term selection (as challenging as it may be) to only match when the value pattern allows it to be expressed using this term, otherwise a more appropriate term could be selected.
So, I think using
@included
seems like the best path forward of the two options. I think frame authors should either focus on creating frames that ensure@included
won't surface or be ready to handle/reject results where it does.
👎
Also, I didn't quite understand this comment (emphasis mine):
We could also put
@included
within the first/arbitrary objectI may be misunderstanding what you're saying here, but it wouldn't be arbitrary nor would I think it would affect compaction because
@included
would be in the expanded output produced during framing.So we'd get something like this from framing:
// top level "@graph": [{ // will be come a `@graph: @container` // post compaction "ex:proof": [{ // first blank node named graph "@graph": [{ // two matches at the graph root level "@included": [{ "@id": "ex:foo1", "@type": ["ex:Foo"] }, { "@id": "ex:foo2" "@type": ["ex:Foo"] }] }] }, { // second blank node named graph "@graph": [{ // just one match at the graph root level "@id": "ex:bar1", "@type": ["ex:Bar"] }] }] }]
Compaction would just run normally on that and produce:
// top level "@context": { "proof": {"@id": "ex:proof", "@container": "@graph"} }, "@graph": [{ "proof": [{ // first blank node named graph // two matches at the root graph level "@included": [{ "@id": "ex:foo1", "@type": "ex:Foo" }, { "@id": "ex:foo2" "@type": "ex:Foo" }] }, { // second blank node named graph // just one match at the root graph level "@id": "ex:bar1", "@type": "ex:Bar" }] }]
So, putting @embedded
in the first object would be reproducible if the ordered option is used, so in your case above, the result could be the following:
// top level
"@context": {
"proof": {"@id": "ex:proof", "@container": "@graph"}
},
"@graph": [{
"proof": [{
// First node exists as value of _proof_
"@id": "ex:foo1",
"@type": "ex:Foo"
"@included": [{
// Second node _included_ within first node.
"@id": "ex:foo2"
"@type": "ex:Foo"
}]
}, {
// second blank node named graph
// just one match at the root graph level
"@id": "ex:bar1",
"@type": "ex:Bar"
}]
}]
This is more in keeping of the sprit of @included
, IMHO.
So, framing would handle injecting
@included
based on the same named graph where multiple matches occurred. So, if you're inside a named graph and you're at the "root level" of that named graph, and we find more than one match, then we have to wrap the matches in an object with a key of@included
and a value that is the array of matches -- and use that as the single value for the named graph@graph
array. With just one match, we use it directly as the value for the named graph@graph
array. If this is what you meant then we're on the same page. But it seems to me like this wouldn't be a modification to compaction at all.
If we were to do so, this step would most likely be handled by compaction, as framing just results in an expanded output, in which there is no @container: @graph
to deal with, and arbitrarily inserting @included
would not mesh well with other compaction strategies.
I think the mechanism I showed is more natural, but unless ordering, the selection of the first node may be arbitrary, but this is a corner-case. This could be something done in the compaction algorithm, when the term has @container: @graph
and there are multiple values, but as I said above, I think this just shouldn't be a match.
In fact, maybe we could get away with always outputting
@included
for named graph matches and letting the compaction step of framing remove it when there's just one node (does compaction do that now?). I don't know what would be easier/more efficient, just putting that out there.
In expanded form, named graphs are serialized like the following:
{
"@graph": [{
"ex:proof": [
{"@id": "_:b1"},
{"@id": "_:b2"}
]
}, {
"@graph": [{
"@id": "ex:foo2",
"@type": "ex:Foo"
}, {
"@id": "ex:foo1",
"@type": "ex:Foo"
}]
}, {
"@graph": [{
"@id": "ex:bar1",
"@type": "ex:Bar"
}]
}]
}
This falls out of the current way RDF would serialize quads to JSON-LD. Of course, you could always frame with @embed
to if that's the shape you want.
- It seems we should not add a node at the top level if it is embedded, but this creates a processing order dependency.
I thought
@once
covered most of these cases already -- and that we didn't add something to the top level unless it matched the frame up there. If it matches at the top level and elsewhere, I don't see why we'd exclude it at the top level in particular. That seems like it would have to be a different@embed
mode -- and that it would introduce more complexity. Do we really need to do anything here?
The issue came to light when creating a trivial frame with @embed: @never
in the top-level of a named graph. In that case, I think it would indicate that all nodes should be created directly under @graph
, and only node references embedded, but the algorithm didn't handle that, as it doesn't know the difference between the top-level and something embedded.
With @embed: @once
, the implication is that it should not appear at the top-level if it is embedded, rather than serializing a node reference at the top-level. The wording of 4.3, which handles @once
is a bit odd: "... and parent has an existing embedded node in parent associated with graph name ...". I think it means that if the node has already been serialized in the graph, output a node reference, but this doesn't not consider if it's at top-level.
The embedded flag
would seem to be necessary to record if we're in an embedding state, or not.
But we have many cases where you can't compact using a term which doesn't match. I think it's in perfect keeping with the spirit of term selection (as challenging as it may be) to only match when the value pattern allows it to be expressed using this term, otherwise a more appropriate term could be selected.
Can I get clarity on what you think should be a match and what shouldn't? For example:
Would this roundtrip (proof
would be a match for the value)?
"@context": {
"proof": {"@id": "ex:proof", "@container": "@graph"}
},
"proof": {
"@id": "ex:foo1",
"@type": "ex:Foo"
"@included": {
"@id": "ex:foo2"
"@type": "ex:Foo"
}
}
What about this?
"@context": {
"proof": {"@id": "ex:proof", "@container": "@graph"}
},
"proof": {
"@included": [{
"@id": "ex:foo1",
"@type": "ex:Foo"
}, {
"@id": "ex:foo2"
"@type": "ex:Foo"
}]
}
If you think both of these should be matches, can you give an example of something that wouldn't roundtrip? If you think the first should round trip but not the second, don't you think that will be confusing/inconsistent? I would think it would special case @included
and make it harder to use to represent "bushes" since there are corner cases to consider.
I'd rather people framing things have to consider cases where their specified matches might produce multiple results at the root level of a graph. That's already a thing they have to do with the default graph when @container: @graph
isn't involved.
It seems to me that the rule for whether or not @container: @graph
is a match is just if a value uses a simple blank node named graph. That also matches the definition/use for the feature. Why would a user expect a non-match just because a particular blank node named graph had multiple roots?
The issue came to light when creating a trivial frame with @embed: @never in the top-level of a named graph. In that case, I think it would indicate that all nodes should be created directly under @graph, and only node references embedded, but the algorithm didn't handle that, as it doesn't know the difference between the top-level and something embedded.
With @embed: @once, the implication is that it should not appear at the top-level if it is embedded, rather than serializing a node reference at the top-level. The wording of 4.3, which handles @once is a bit odd: "... and parent has an existing embedded node in parent associated with graph name ...". I think it means that if the node has already been serialized in the graph, output a node reference, but this doesn't not consider if it's at top-level.
The embedded flag would seem to be necessary to record if we're in an embedding state, or not.
:+1:
But we have many cases where you can't compact using a term which doesn't match. I think it's in perfect keeping with the spirit of term selection (as challenging as it may be) to only match when the value pattern allows it to be expressed using this term, otherwise a more appropriate term could be selected.
Can I get clarity on what you think should be a match and what shouldn't? For example:
Would this roundtrip (
proof
would be a match for the value)?"@context": { "proof": {"@id": "ex:proof", "@container": "@graph"} }, "proof": { "@id": "ex:foo1", "@type": "ex:Foo" "@included": { "@id": "ex:foo2" "@type": "ex:Foo" } }
Yes, because when selecting "proof", you have a value with just a single node definition. The contents of @included
isn't considered.
What about this?
"@context": { "proof": {"@id": "ex:proof", "@container": "@graph"} }, "proof": { "@included": [{ "@id": "ex:foo1", "@type": "ex:Foo" }, { "@id": "ex:foo2" "@type": "ex:Foo" }] }
Yes, that would match too, because there is still just a single member (@included: [...]
).
If you think both of these should be matches, can you give an example of something that wouldn't roundtrip? If you think the first should round trip but not the second, don't you think that will be confusing/inconsistent? I would think it would special case
@included
and make it harder to use to represent "bushes" since there are corner cases to consider.
Look how each expands:
[{
"ex:proof": [{
"@graph": [{
"@id": "ex:foo1",
"@type": ["ex:Foo"],
"@included": [{
"@id": "ex:foo2",
"@type": ["ex:Foo"]
}]
}]
}]
}]
and
[{
"ex:proof": [{
"@graph": [{
"@included": [{
"@id": "ex:foo1",
"@type": ["ex:Foo"]
}, {
"@id": "ex:foo2",
"@type": ["ex:Foo"]
}]
}]
}]
}]
If, instead, you had the following expanded:
[{
"ex:proof": [{
"@graph": [{
"@id": "ex:foo1",
"@type": ["ex:Foo"]
}, {
"@id": "ex:foo2",
"@type": ["ex:Foo"]
}]
}]
}]
it would not select "proof", as the value to compact has two members. You can address this in framing through the use of @included
.
I'd rather people framing things have to consider cases where their specified matches might produce multiple results at the root level of a graph. That's already a thing they have to do with the default graph when
@container: @graph
isn't involved.It seems to me that the rule for whether or not
@container: @graph
is a match is just if a value uses a simple blank node named graph. That also matches the definition/use for the feature. Why would a user expect a non-match just because a particular blank node named graph had multiple roots?
Well, I won't stand in the way of this, but in that case, I'd say that the compaction algorithm should insert the @included
in the first node definition member with the remaining nodes to yield the following:
{
"@context": {
"@version": 1.1,
"proof": {"@id": "ex:proof", "@container": "@graph"}
},
"proof": {
"@id": "ex:foo1",
"@type": "ex:Foo",
"@included": {
"@id": "ex:foo2",
"@type": "ex:Foo"
}
}
}
Note that this discussion is really about https://github.com/w3c/json-ld-api/issues/143. If we agree what the framing algorithm should do, as indicated by this PR, we can close it.
The only change I'll make is to also consider values of @included
to be top-level nodes and set embedded flag
to false
.
Should have a formal resolution to close.
This issue was discussed in a meeting.
RESOLVED: Close api #143 as resolved by api PR #145 and #146
@container
: @graph
got mangled in re-expansion@included
block
I'm trying to create an "identity frame" for some data that uses
"@container": "@graph"
. The data looks like this:And the frame like this:
A link to the playground is here: http://tinyurl.com/yaewmkk5
The goal is for the output to match the input, but it does not. Does anyone know what tweaks are necessary to accomplish this? Do we have proper support for framing blank node named graphs? From looking at the jsonld.js implementation, it seemed like only non-blank node named graphs were supported... and the spec doesn't seem to give any examples with blank nodes. It seems we have both a spec and an implementation gap here. If not, could someone share a working example?