search

knocean / knode

Knowledge Development Environment
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

Implement Linked Data Fragments server #62

Open jamesaoverton opened 6 years ago

jamesaoverton commented 6 years ago

The next feature that I want is a Linked Data Fragments server http://linkeddatafragments.org. Most of it should be implemented in CLJC, and we can refer to other implementations. This is just the dumb server part that returns quads/triples, not the more clever client part that handles SPARQL queries.

The backing data structure will be a vector of maps from keywords to strings or nil, representing an RDF quad. The keys are:

I've attached a sample file with ~3000 maps: obi_core.edn.zip

The system should accept queries like http://fragments.dbpedia.org/2016-04/en?predicate=http%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23label&object=%22Brad%20Pitt%22%40en and return HTML pages very similar to that one. The basics are straightforward. Implementing all the details will take some work, but they're important.

We can start with triples, then look at quads (which LDF doesn't seem to support as well yet).

The goal for this first part is to be serve results that work for the primary LDF client implementation: https://github.com/LinkedDataFragments/Client.js

The main motivation for the new flat map representation is that it's easy to write to and read from a SQL table using JDBC. Once the basics are implemented in CLJC, I'd like a CLJ implementation that's backed by SQL. We can start with in-memory SQLite, then SQLite on disk, then connect to PostgreSQL.

The goal for this second part is to return an HTML page with ~100 triples in less than a second, even when the SQL table has ~10 million rows.

inaimathi commented 6 years ago

Work on this happening at https://github.com/knocean/knode/tree/linked-data-fragments

inaimathi commented 6 years ago

Work proceeds in https://github.com/knocean/knode/tree/linked-data-fragments We've got the main functionality working in SQL; my next step is a cleanup phase wherein I'll separate out the SQL-backed implementation from the basic one and re-factor some of the request/query translation pieces. Once that's done, the last bit I'll need to finish is support for blank nodes in queries.

inaimathi commented 6 years ago

Complete. This works with Sqlite3 in-memory and on-disk (and should in theory work transparently for Postgres with the appropriate driver and DB spec).