search

Staatsarchiv-Basel-Stadt / RDF-Pipeline

RDF-Pipeline Staatsarchiv Basel-Stadt
0 stars 0 forks source link

Refactor Stardog virtual graph updates #18

Closed ktk closed 2 years ago

ktk commented 2 years ago

We need to refactor this script in the repo: https://github.com/Staatsarchiv-Basel-Stadt/RDF-Pipeline/blob/master/shell/mappingUpdate.js

This script copies the scope mapping from this repo to the Stardog server. This is all done via scp which does not make much sense as there is an API from Stardog for that.

In other words it should be refactored to this API instead so shell access is no longer necessary.

What should remain is the regular pulling for updates in the StABS-scope2RDF repo. The script will be run via cron.

ktk commented 2 years ago

I forgot to mention there is also stardog.js: https://github.com/stardog-union/stardog.js

oschihin commented 2 years ago

@ludovicm67 set the mapping update to the new development branch of the repo

ludovicm67 commented 2 years ago

Done, but it needs to be reviewed to check if everything is OK.

ktk commented 2 years ago

@ludovicm67 will update the code & test with a 3rd party repo

ludovicm67 commented 2 years ago

I did a lot of testing locally on my machine using a PostgreSQL database. The script is working as expected. Issues on "production" may come from issues from the Oracle JDBC driver (someone can try to change the version in the stardog-docker project), bad credentials or issues with proxies somewhere.

More details here about my local test: https://hackmd.io/@ludovicm67/zazuko-basel-mapping

From my point of view, the script is OK and the work I needed to do is done. Since I'm not able to do any further testing, someone else than me should take care of checking the credentials, the driver and the proxies' configuration.

oschihin commented 2 years ago

@ludovicm67 thanks a lot for your work. I adapted some local stuff (ports, volumes) and all is up running internally. Haven't tested against oracle yet, that's for @ktk next week