Linked Data Fragments - Githubissues

akuckartz commented 10 years ago

Bei Linked Data Fragments (http://linkeddatafragments.org/) geht es darum, ein praktisches Problem von SPARQL zu lösen, nämlich die sehr ungleiche Belastung von Servern in Richtung Clients zu verschieben. Potentiell können auf der Grundlage die bisher fehlenden OParl-Abfragemöglichkeiten zur Verfügung gestellt werden.

Issues die damit (zumindest teilweise) erledigt werden könnten: #203, #200, #175, #173, #167.

akuckartz commented 10 years ago

Der Beispiel-LDF Client http://client.linkeddatafragments.org/ kann auch SPARQL-Queries ausführen, die FILTER enthalten:

PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>

SELECT * WHERE {
  ?country a dbpedia-owl:Country;
           <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> ?area.
  FILTER (xsd:double(?area) > 1000)
}

Ich habe noch nicht ausprobiert, ob das auch bereits mit xsd:date funktioniert - was für OParl wichtig ist. Für die Einschätzung der Performance wären dann auch genauer herauszufinden, was davon auf dem Server und was im Client passiert.

akuckartz commented 10 years ago

Siehe https://github.com/LinkedDataFragments/Client.js/issues/5

marians commented 10 years ago

Ich habe mir nun die Informationen auf http://linkeddatafragments.org/ grob durchgesehen. Mein Eindruck ist: Das System ist komplex und damit schwer verständlich. Damit ist die Aussicht, die Vor- und Nachteile des Konzepts bis zu unserer Deadline verstanden und daraus eine belastbare Empfehlung gemacht zu haben, gering.

Deshalb spreche ich mich dafür aus, das Thema Linked Data Fragments aus Version 1.0 komplett heraus zu halten.

akuckartz commented 10 years ago

@marians Es geht mir nicht um eine Festlegung in Version 1.0 sondern zunächst nur um einen Ausblick in die (nicht allzu ferne) Zukunft, da verständlicherweise Bedarf nach mehr Abfrage- bzw. Filtermöglichkeiten geäußert wurde. Linked Data Fragements ist einer der Ansätze mit denen dies möglicherweise angegangen werden kann.

Auch wenn das Gesamtkonzept nicht trivial ist, so kann eine Implementierung auf Server-Seite durchaus ziemlich einfach aussehen. "basic Linked Data Fragments" sind im Kern schlicht Filter, bei denen ein oder zwei der Tripel-Positionen (Subjekt, Prädikat, Objekt) festgelegt sind.

akuckartz commented 10 years ago

Hinweis in Abschnitt 1040 aufgenommen.

RubenVerborgh commented 10 years ago

@marians I'm the lead developer of the Linked Data Fragments client.

Would you like to get in touch? It's not as complex as it seems, quite the opposite :wink:

akuckartz commented 10 years ago

@RubenVerborgh I have read some of the papers explaining what Linked Data Fragments is about and agree that the parts relevant for the OParl specification are not very complex. The problem is that we only have a few days left until we have planned to finalize OParl 1.0.

My idea is that it will be possible to write a supplemental document after that which specifies how Linked Data Fragments can be used to provide and access Oparl data. We do not need to wait for OParl 1.1 or OParl 2.0 to do that - and there will be demand for some kind of query language for OParl. basic LDF is the most promising simple solution I am aware of. I suppose we will also find a concrete solution for this issue regarding date ranges: https://github.com/LinkedDataFragments/Client.js/issues/5

BTW: We are also interested in full text search. Maybe LDF can even support that. Then we probably would have everything we need for a query language ;-)

RubenVerborgh commented 10 years ago

That's really cool. Let's indeed try to do something soon (after 1.0 has been released then). Do you have a current RDF interface somehow?

Fulltext: one of the extensions to basic LDFs are indeed fragments that support text search. OParl would be a very nice use case for that!

PS Here is public data from the Belgian government: data.kbodata.be. Different kind of data though, but it shows the potential for such things.

akuckartz commented 10 years ago

@RubenVerborgh Is a current specification of basic LDF available anywhere? We would need that for an OParl supplement. I understand how the URL parameters subject, predicate and object are used. But is the format of the response specified? I understand that Hydra likely will be used in the future, but would like to know how the format looks like at the moment.

RubenVerborgh commented 10 years ago

@akuckartz Literally working on it as we speak. Should be out by next week.

The response needs to contain:

the (first X) triples with the given subject, predicate, and object
the estimated total number of triples with the given subject, predicate, and object (void:triples)
a (Hydra) form that explains how clients can find other fragments (i.e., the URL template)

An example of 2:

<http://data.linkeddatafragments.org/dbpedia?object=dbpedia%3AGermany>
    void:triples 182686;
    hydra:totalItems 182686;
    hydra:itemsPerPage 100.

An example of 3:

<http://data.linkeddatafragments.org/dbpedia#dataset>
    void:subset <http://data.linkeddatafragments.org/dbpedia?object=dbpedia%3AGermany>;
    hydra:search [
        hydra:template "http://data.linkeddatafragments.org/dbpedia{?subject,predicate,object}";
        hydra:mapping [
          hydra:variable "subject";
          hydra:property rdf:subject
        ], [
          hydra:variable "predicate";
          hydra:property rdf:predicate
        ], [
          hydra:variable "object";
          hydra:property rdf:object
        ]
    ].

So note that all we need is just to machine equivalent of what a human also sees. I.e., if you look at the fragment I described above, you will see that humans also see the total count, and also have a form to find other fragments. That's all there is too it (but then in RDF).

akuckartz commented 10 years ago

@RubenVerborgh Content negotiation works, great!

curl -H "Accept: application/ld+json" http://data.linkeddatafragments.org/dbpedia?object=dbpedia%3AGermany

RubenVerborgh commented 10 years ago

@akuckartz Glad you noticed! Yes, an important aspect of our approach is that human and machine clients have the same interface. This is reflected in the use of a single fragment URL for all clients (which, through content negotiation, each get their own representations). Furthermore, each of the representations contains all elements necessary for clients. For instance, human clients get an HTML form to navigate through fragments, machine clients get a Hydra form (cfr. my comment above).

So a basic Linked Data Fragments interface is a REST interface in the actual sense of the term (resources / representations / self-describing messages / hypermedia).

akuckartz commented 10 years ago

@RubenVerborgh I noticed a detail which I do not understand. Why is the middle one of these three JSON-objects included in the output? The object of the triple is not dbpedia:Germany:

    {
      "@id": "dbpedia:Ford_Transit",
      "dbpedia-owl:assembly": {
        "@id": "dbpedia:Germany"
      }
    },
    {
      "@id": "dbpedia:Germany",
      "rdfs:seeAlso": {
        "@id": ":dbpedia?subject=http%3A%2F%2Fdbpedia.org%2Fresource%2FGermany"
      }
    },
    {
      "@id": "dbpedia:Gholam_A._Peyman",
      "dbpedia-owl:almaMater": {
        "@id": "dbpedia:Germany"
      }
    },

RubenVerborgh commented 10 years ago

@akuckartz A basic Linked Data Fragment contains data, metadata, and hypermedia controls. That means that, in addition to triples matching the pattern, there are triples that capture the metadata and the controls. The particular triple you highlight is part of the metadata.

This might be a little more clear in the Turtle representation:

# Controls
<http://data.linkeddatafragments.org/dbpedia#dataset> a void:Dataset, hydra:Collection;
    void:subset <http://data.linkeddatafragments.org/dbpedia?object=dbpedia%3AGermany>;
    void:uriLookupEndpoint "http://data.linkeddatafragments.org/dbpedia{?subject,predicate,object}";
    hydra:search _:triplePattern.
_:triplePattern hydra:template "http://data.linkeddatafragments.org/dbpedia{?subject,predicate,object}";
    hydra:mapping _:subject, _:predicate, _:object.
_:subject hydra:variable "subject";
    hydra:property rdf:subject.
_:predicate hydra:variable "predicate";
    hydra:property rdf:predicate.
_:object hydra:variable "object";
    hydra:property rdf:object.

# Metadata
<http://data.linkeddatafragments.org/dbpedia?object=dbpedia%3AGermany> a hydra:Collection, hydra:PagedCollection;
    dcterms:title "A 'dbpedia' Linked Data Fragment"@en;
    dcterms:description "Basic Linked Data Fragment of the 'dbpedia' dataset containing triples matching the pattern { ?s ?p http://dbpedia.org/resource/Germany }."@en;
    dcterms:source <http://data.linkeddatafragments.org/dbpedia#dataset>;
    hydra:totalItems "182686"^^xsd:integer;
    void:triples "182686"^^xsd:integer;
    hydra:itemsPerPage "100"^^xsd:integer;
    hydra:firstPage <http://data.linkeddatafragments.org/dbpedia?subject=&predicate=&object=http%3A%2F%2Fdbpedia.org%2Fresource%2FGermany&page=1>;
    hydra:nextPage <http://data.linkeddatafragments.org/dbpedia?subject=&predicate=&object=http%3A%2F%2Fdbpedia.org%2Fresource%2FGermany&page=2>;
    dcterms:subject dbpedia:Germany.
dbpedia:Germany rdfs:seeAlso <http://data.linkeddatafragments.org/dbpedia?subject=http%3A%2F%2Fdbpedia.org%2Fresource%2FGermany>.

# Data
dbpedia:University_of_Tirana dbpedia-owl:affiliation dbpedia:Germany.
dbpedia:Michael_Ristow dbpedia-owl:almaMater dbpedia:Germany.
dbpedia:BMW_5_Series dbpedia-owl:assembly dbpedia:Germany.
dbpedia:Maybach_Zeppelin dbpedia-owl:assembly dbpedia:Germany.
dbpedia:Erkki_Nghimtina dbpedia-owl:almaMater dbpedia:Germany.
dbpedia:Audi_S6 dbpedia-owl:assembly dbpedia:Germany.
dbpedia:Raziuddin_Siddiqui dbpedia-owl:almaMater dbpedia:Germany.
# …

Note that this particular piece of metadata is optional for basic LDFs; only count metadata is mandatory.

marians commented 10 years ago

Thanks @RubenVerborgh for joining our discussion! This is all fine with me as long as the issue remains tagged "future" (which means "beyond 1.0").

akuckartz commented 10 years ago

Die dazu erforderliche Ergänzung der OParl-Spezifikation wird sich weitgehend auf Verweise auf diese Dokumente beschränken können: http://www.hydra-cg.com/spec/latest/triple-pattern-fragments/ http://www.hydra-cg.com/spec/latest/linked-data-fragments/

akuckartz commented 10 years ago

Eine Behandlung von LDFs für Datumsbereiche wird erarbeitet: https://github.com/HydraCG/Specifications/issues/56

akuckartz commented 10 years ago

@RubenVerborgh I am closing this issue but will contact you separately.

RubenVerborgh commented 10 years ago

Okay, thanks!

OParl / spec

Linked Data Fragments #165