blazegraph / database

Blazegraph High Performance Graph Database
GNU General Public License v2.0
873 stars 170 forks source link

Setting up HAJournalServer - can it be done? #116

Open amp4045 opened 5 years ago

amp4045 commented 5 years ago

Sadly this isn't an issue, more of a question:

has anyone ever had luck setting up the HAJournalServer? It lacks much documentation. Thank you for your time and patience, hope to hear from someone soon!

amp4045 commented 5 years ago

I tried to follow the documentation that currently exists for the basic deployment:

Downloading the source code, running "ant deploy-artifact". Turns out this has changed and really you need to execute "ant maven-deploy". Unfortunately, the deploy argument of the mvn command fails for me on the maven-installer-plugin dependency.

Has anyone out there had any success?

amp4045 commented 5 years ago

Updates: I've also tried running the new maven script..

./scripts/mavenInstall.sh

but that didn't create any artifacts/builds that contained the journal config files I was pointed to modify in the docs.

Any information is helpful :)

thompsonbry commented 5 years ago

The HA configuration is not functional in more recent releases. Systap halted development of the Blazegraph HA feature several years ago (long before we came to Amazon). Full HA is a complex thing to develop and maintain with master failure, testing of the various failover configurations, longevity testing, targeted failure mode tests, etc. We self-funded quite a bit, but we did not get the engagement from the open source community to make it worth while to continue HA as an open source feature.

You can always do the poor man's HA, put the updates onto a durable queue, and then apply writes to each server in parallel. You would need to handle master failover of course. Or you can capture the IChangeLog from one server and replicate the post-facto changes (in terms of statements added and removed) to the other servers, again using a durable queue to capture the post-commit change set. To do the latter, you would also need to report additions to the dictionary indices (which is not currently done, but which would not be that difficult to add in the LexiconRelation and an apply loop interface for the replicas to apply the deltas on their local indices). I think this might "just work". The local journal tracks the transactions in flight and manages the recycling of deleted records once no transaction can read on those records. So transactional access to data should "work" on the replicas without doing anything else. Again, you would need to handle master failover, etc.

Thanks, Bryan

On Mon, Feb 18, 2019 at 10:30 AM Andrew Peacock notifications@github.com wrote:

Updates: I've also tried running the new maven script..

./scripts/mavenInstall.sh

but that didn't create any artifacts/builds that contained the journal config files I was pointed to modify in the docs.

Any information is helpful :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/blazegraph/database/issues/116#issuecomment-464836869, or mute the thread https://github.com/notifications/unsubscribe-auth/ACdv4Ea1uniFenR4U_hRmSJA5oX5o96Zks5vOvE4gaJpZM4a-cEJ .

c5haw commented 4 years ago

I am following up on this as I am trying to ascertain if HAJournalServer is possible. Is this still in the same state, i.e. that there is no official development in progress at this stage?

thompsonbry commented 4 years ago

Nothing has changed since the posting above

Thanks, Bryan

On Mon, Jun 8, 2020 at 11:01 Chris Shaw notifications@github.com wrote:

I am following up on this as I am trying to ascertain if HAJournalServer is possible. Is this still in the same state, i.e. that there is no official development in progress at this stage?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/blazegraph/database/issues/116#issuecomment-640783844, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATW7YFJZLSJXHPGAQJBYY3RVURQ7ANCNFSM4GXZYEEQ .