search

linkeddata / rdflib.js

Linked Data API for JavaScript
http://linkeddata.github.io/rdflib.js/doc/
Other
566 stars 146 forks source link

Implement RDFJS Representation Task Force spec #137

Open dmitrizagidulin opened 8 years ago

dmitrizagidulin commented 8 years ago

Update rdflib.js to match the RDFJS Representation Task Force spec. Road map:

Core (Terms, Quads, Graphs)

(In progress - see DataSet spec wiki page)

Stream interfaces
timbl commented 8 years ago

The Turtle/N3 serializer cannot have a stream interface, as it takes the statements and sorts them before outputting them, to get a compact and pretty version.

RubenVerborgh commented 8 years ago

The interface can still be streaming: the writer is free to buffer it all and only output at the end.

In other words: a non-streaming writer can be realized with a streaming interface; but a streaming writer can only be realized with a streaming interface. Hence, RDF/JS chooses the most generic option by defining a stream interface.

elf-pavlik commented 8 years ago

I think in many use cases the 'pretty' requirement may not matter. Of course for people doing development, tools need to exist to pretty print some output, for CLI use one can always use pipes for that and modern (extensible) code editors often provide 'pretty format' plugins.

With progress on RDF/JS soon developers will have possibility to pick parsers, serializers and stores that they find fitting their requirements best. Some applications will use serializers that offer pretty output while others may choose better performing serializer that provides valid but not prettified output. I hope soon participants of RDF/JS will start documenting differences between various implementations of modules with similar functionality, this should assist people to choose which combination of modules may meet their requirements the best. Graphy.js has IMO nice example of comparing it's own parser to the one in N3.js https://github.com/blake-regalia/graphy.js#performance

elf-pavlik commented 8 years ago

@dmitrizagidulin as @bergos mentioned today, he already made available some parsers and serializers that implement RDF/JS stream interface: https://github.com/rdfjs/representation-task-force/wiki/Modules#done

Maybe trying to use rdf-parser-jsonld in rdflib.js would make a good start?

I noticed

const N3 = require('n3') // @@ Goal: remove this dependency

in both

I guess with rdf-serializer-jsonld you will not need to use N3.js any more to write N-Quads and convert them to JSON-LD with jsonld.js? But it looks like you still need N-Quads serializer in https://github.com/linkeddata/rdflib.js/blob/master/src/serialize.js#L37

elf-pavlik commented 8 years ago

This almost worked in parse.js but rdflib.js graph doesn't seem to have an easy way to add quads with minimal RDF/JS interface

const stringToStream = require('string-to-stream')
const parsers = {
  'application/ld+json': require('rdf-parser-jsonld')
}

/// ...

else if (contentType === 'application/ld+json') {

      let parser = new parsers[contentType](stringToStream(str))
      parser.on('data', function(quad) { return kb.add(quad) })
      parser.on('end', executeCallback)

    }

when I passed [] instead kb.push(quad)worked just fine

possibly passing a factory to parser would allow creating quads with extended interface https://github.com/rdfjs/representation-task-force/issues/79

HolgerKnublauch commented 7 years ago

I noticed that parsing RDF lists (with Turtle) produces a Collection node, which then does not comply to standard RDF when queried via statementsMatching. I believe there should be at least a mode in which rdf:Lists are treated according to standard rdf:first/rest triples. Collections could still remain as a convenience layer, produced on the fly.

dmitrizagidulin commented 7 years ago

@HolgerKnublauch Thanks for bringing this to our attention! Do you mind opening an issue, for Collection nodes & querying, specifically?

elf-pavlik commented 7 years ago

@dmitrizagidulin @RubenVerborgh any plans to add stream interfaces, at least for start to parsers? rdf-ext has both json-ld and n3 wrapped

once N3.js implements http://rdf.js.org stream interface it could get switched...

having both rdf-ext and rdflib.js implementing RDF/JS would make an awesome improvement in the ecosystem

RubenVerborgh commented 7 years ago

Yes, hope to get to that soon.

elf-pavlik commented 6 years ago

@dmitrizagidulin RDF/JS interface already has few completed implementations, you can see list in https://github.com/rdfjs/representation-task-force/issues/67

It includes N3.js just announced by @RubenVerborgh and migrated to @rdfjs organization. Can you think of a way how people who already implemented RDF/JS stream interfaces could help with their experience to complete implementation in rdflib.js ?

elf-pavlik commented 6 years ago

Seeing #243 reminds me or a conversation some years ago about rdflib.js relying on side effects to populate the store. TimBL mentioned that this way indexing can work more effective since you don't have to do it once when you parse and other one when you add it to the store. Still I would find it great if rdflib.js implemented RDFJS stream interfaces and possibly start deprecating passing store as argument to parse. Since it relates to indexing the store it also seems related to #238

timbl commented 4 years ago

The task of providing a RDF/JS compatible interface now becomes in a way crisper when we use Typescript. Before, we could happily exchange objects of any type so long as they had the right properties. And some of the RDFJS folks has suggested accepting plain objects with no classes. Now though with typescript there is the task @megoth and I have been discussing about how you allow the basic RDFJS world and the more powerful rdflib world coexist so that developers can gain the maximum benefit. Even though the rdflib world has been written to be more accepting for example of quads having a literal as a subject, it can be used in systems which don't do that. A system which can handle literals and subgraphs as subjects should be able to use parsers which don't. We need to be able surface both interfaces in typescript, the lower level RDFJS and the higher level rdflib one.

timbl commented 4 years ago

See also https://github.com/linkeddata/rdflib.js/issues/374

RubenVerborgh commented 4 years ago

The task of providing a RDF/JS compatible interface now becomes in a way crisper when we use Typescript.

Not really.

Before, we could happily exchange objects of any type so long as they had the right properties.

We still can; TypeScript is just a mechanism to check the presence of those properties at compile time.

angelo-v commented 3 years ago

The toCanonical method has been removed from the spec, but rdflib still relies on it