search

levelgraph / levelgraph-jsonld

The Object Document Mapper for LevelGraph based on JSON-LD
113 stars 16 forks source link

Handle named graphs #43

Open jmatsushita opened 7 years ago

jmatsushita commented 7 years ago

With the recent version update, it is now possible to submit JSON-LD docs which includes an @graph key. However, named graphs are not handled yet. (More precisely, if there is a top level @id then the @graph value is just ignored).

Given the discussion on mcollina/levelgraph#43 I'm thinking of first implementing this using a graph triple property for put, and a slow filter JS-land approach for get.

The performance penalty will be big on anything that tries to use named graphs but it's probably good to start here before doing anything more involved. Things will probably take shape as I implement but does this make sense @mcollina ?

jmatsushita commented 7 years ago

To clarify, I'm not looking at the sublevel approach yet because I'm interested in provenance cross graph queries and graph per resource or graph per aspect patterns

mcollina commented 7 years ago

@jmatsushita I have not so much time to dig into this, so I would leave it to you. As a general rule, start simple and then optimize. I think the filter should happen as soon as possible in search() at the clause level. In that way we can get thing faster. Another option might be to write levelsubgraph, as a thing that maps multiple levelgraphs based on the graph. Use https://www.npmjs.com/package/subleveldown and not sublevel.

jmatsushita commented 7 years ago

Thanks for the input. I'll document my progress here.

jacoscaz commented 7 years ago

@jmatsushita @mcollina we've just published the first proof of concept of node-quadstore, an attempt at using LevelGraph's LevelDB-based storage approach with a quad-oriented indexing strategy to implement a graph database with native support for cross-graph queries. It still uses 6 indexes. As we do not want to re-invent the wheel, assuming our solution works out we'd be more than happy to contribute to LevelGraph in any way you might deem fit.

mcollina commented 7 years ago

feel free to send a PR to LevelGraph if you want to! :)

BigBlueHat commented 7 years ago

Thinking through @jmatsushita's work in #54--which you can demo in a fork of the playground--it's not possible to keep the provenance of a single triple if it exists (or arrives via) two different named graphs.

Try the example in that playground fork, and change 73 to 74...there will be a triple left behind so it's clear that 73 was inserted, but you'd not be able to re-compute the @graph as all its references are lost relative to the triples--all being replaced by the identical graph with different name (i.e. 74).

If the graph property were stored as an array (or object for easier uniqueness), that would solve for this issue...but I'm not sure it'd scale so great...

Thoughts? 💭

jmatsushita commented 7 years ago

Hi @BigBlueHat thanks for your continued interest! Indeed this is a bug, or a design shortcoming of my fork, in the sense that it cannot work with levelgraph's current indexing strategy. Basically the graph property is just metadata added to triples so 2 triples with a differing graph are in fact the same triple for levelgraph, hence the behavior you see on the playground.

There would need to be a change (or an option) in levelgraph in order for quads to be indexed. There was some chat about this on the levelgraph issue tracker https://github.com/mcollina/levelgraph/issues/43

It seems that node-quadstore manages to do quads with 6 indices and now is aligning with recent work on RDF/JS. It also has a LevelUp compatible storage backend and its Graph API (not the RDF/JS one) is fairly close to the LevelGraph one. So it should be possible to write a quadstore-jsonld module, or even better a rdfjs-jsonld module.

As @elf-pavlik suggested in https://github.com/mcollina/levelgraph/issues/43 (and if quad indexing is added into levelgraph) creating a levelgraph-rdf layer would mean that more robust and interoperable approaches to levelgraph-jsonld and levelgraph-n3 would be able to exist. Adding something like clownface would also allow something very similar to navigation API from LevelGraph as well.

Given all of this recent good work I would be inclined to watch that issue mcollina/levelgraph#43 for possible addition of quad indexing into levelgraph and consider my fork of levelgraph-jsonld as a deadend. I will also definitely give a try to @jacoscaz's work on node-quadstore which looks very promising!

jacoscaz commented 7 years ago

@jmatsushita @BigBlueHat I think a levelgraph-rdf layer would make the most sense as it would allow levelgraph to become a part of the growing RDF/JS ecosystem. A rdfjs-jsonld module would then be a wonderful contribution to the whole ecosystem. We ourselves are thinking about extracting the query layer of node-quadstore into a dedicated rdfjs-* module.

BigBlueHat commented 7 years ago

I'm definitely down for more collaboration! 😁 I'm already lurking in all the places. :wink: Exciting times ahead!