Multi-User-Domain / mud-jena

0 stars 0 forks source link

Abstract Away Dataset Backend #32

Open calummackervoy opened 3 years ago

calummackervoy commented 3 years ago

The bulk of the World Server functionality is looking after a datastore. There are existing solutions for this compatible with RDF data, and I think we could save ourselves effort if we integrated one or both of those:

I did some research on this and both would most easily entail a service running on another port, with MUD Jena communicating to it over HTTP. LinkedDataHub can be queried with SPARQL directly and Neo4J uses its own Cypher query language (it looks very intuitive but it's got nothing to do with semantic standards). Neo4J argues that it has a number of quite crushing advantages over SQL and RDF triplestores in design (see https://neo4j.com/blog/rdf-triple-store-vs-labeled-property-graph-difference/), and that data can be imported/exported to RDF using neosemantics. Neo4J has drivers to loads of languages and an enthusiastic community, I think that long-term providing support for it could be very rewarding

Since LinkedDataHub uses the same stack as us it's probably the easier of the two options as a starting point

We already abstract away from the file locations and TDB connections with our DataSetItem, DataSetCollection and TDBStore classes, a priori this would be another layer of abstraction to that, but I'm open to changing the way those abstractions work as well, I don't think they're perfect by any means

@MattTennison you mentioned that one of the things you liked about this project is getting the decentralised services to talk to each other. This is kind of along those lines maybe but you might not have had databases in mind ? 😅 I guess it has implications for our Docker container, too ?