External Triplestore - Githubissues

ualbertalib / HydraNorth

This repo is deprecated. Succeeded by https://github.com/ualbertalib/jupiter. This codebase was a IR built based on Samvera/Sufia

11 stars 4 forks source link

External Triplestore #139

Closed pbinkley closed 7 years ago

pbinkley commented 9 years ago

As metadata team member, I want the ability to access all the triples of all the Fedora objects in an external triplestore in order to

query the data
run reports on the data
explore data cleanup and normalization
model the use of equivalencies versus transformations ans explore benefits and drawbacks
visualize data to assess potential impact of proposed changes
run reconciliation processes on the data

The triplestore should be

as close a mirror as possible of the triple set in Fedora (production)
fully and easily accessible to the metadata team
configured so that all associated activity poses little or no risk to the system as a whole

See Piyapong's preliminary implementation of Fuseki, OpenRDF and Marmotta. See also #140 .

pgwillia commented 8 years ago

@piyapongch might be interested in a conversation https://groups.google.com/forum/#!topic/ansible-project/2a96TqzVSN0

I have a Karaf setup with fuse-esb, not sure how they are different, but fuse has a "client" script which I use to launch commands. Here is a snippet of my task:

name: Execute Command in Karaf shell: "{{app_dir}}/{{fuse_esb_home}}/bin/client -h localhost -a {{karaf_port}} -u {{karaf_user}} -p {{karaf_password}} {{karaf_command}}"

piyapongch commented 7 years ago

I have installed fcrepo-camel-toolbox-4.5.2 on Karaf, OSGi server on Sheff server. I have successfully indexed the data in Fedora 4.2.0 on Plano to the Fuseki Triplestore on Sheff. I am still working on indexing data from Fedora 4.2.0 on Gillingham that has data close to the production server. Here is my notes about the triplestore work, https://docs.google.com/document/d/1d-yX0aRMhTsygFifHqHVaCkz305o1EsxGTvDmLu-eWA/edit?usp=sharing.

piyapongch commented 7 years ago

Triplestore Works

[x] Install Apache Karaf and fcrepo-camel-toolbox: fcrepo-indexing-tripletore, fcrepo-reindexing, fcrepo-audit and fcrepo-fixity and howtio
[x] Install Apache Jena Fuseki triplestore
[x] Install Tomcat7 and deploy Apache Marmotta, RDF4J triplestores
[x] Install Blazegraph triplestore
[x] Deploy jolokia webapp agent on Tomcat7 and add jolokia agent to Fuseki configuration
[x] Configure fcrepo-reindexing and fcrepo-indexing-triplestore
[ ] Test fcrepo-indexing-triplestore to make sure that automatically update is working properly
[ ] Configure fcrepo-audit and test
[ ] Configure fcrepo-fixity and test
[x] Index Fedora4 data into Fuseki
[x] Index Fedora4 data into RDF4J
[ ] Index Fedora4 data into Marmotta
[x] Index Fedora4 data into Blazegraph
[ ] Setup triplestore production server
[ ] Index Fedora4 production server to selected triplestore
[x] Notes, https://docs.google.com/document/d/1d-yX0aRMhTsygFifHqHVaCkz305o1EsxGTvDmLu-eWA/edit?usp=sharing

weiweishi commented 7 years ago

The decision is to let the metadata team test the triplestores @piyapongch set up, and we will make a decision about the triplestore selection for production in the future. We will split the remaining tasks into new tickets.