search

biblia-arabica / srophe-eXist-app

eXist code for Syriaca.org: The Syriac Reference Portal
GNU General Public License v3.0
0 stars 2 forks source link

Explore website archival options #109

Open nathangibson opened 5 years ago

nathangibson commented 5 years ago

We would like to figure out some more robust ways of archiving the website for both long-term and short-term viability. Can you help us figure out our options and implement the ones that have to do with the Srophe app? (I'm also interested in this for Usaybia, if more support is needed than what BA has available.)

Here are some specific directions I'm thinking.

  1. Archiving the data: I think we have this covered. We plan to periodically archive the TEI files themselves in Zenodo and/or the university library data repository. At the same time, we can archive the eXist packages for the data and Srophe app.

  1. A simple, archive version of the website (HTML5, JS, CSS) as a fallback for eXist failure and a plug-n-play option if the project needs life support from a different server: Martin Holmes presented at TEI 2019 about using static website versions (generated by eXist), see https://doi.org/10.5281/zenodo.3449196. See also https://projectendings.github.io/principles/ and code at https://github.com/projectEndings/. I don't think we want to go as radical as he does, but I do wonder if we could generate static HTML pages for each item detail page, as well as a browse list with links for single facets (not multiple faceting). This version would probably not have search. (Martin Holmes has created a staticSearch engine at the above Github repo, which uses JS only, but it might be too much work to implement for our purposes.) It would be amazing if there's a way to trigger a redirect to the archival page when eXist hits an error on one or to the archival website in general when the server generates an error code like 501 (i.e. eXist is down). This simple website version is also something we could easily archive and could be redeployed easily if we don't have the overhead for an eXist server, etc.

    1. Archiving a docker image of eXist with the apps installed that can easily be installed on another server (alternative to or additional to no. 2 above). I think you were looking into what we can do with Docker Compose? It would be really awesome to periodically be able to save a snapshot, where all you need to do is deploy the Docker image and voila! the BA website is up and running. (Okay, I know you still need Jetty and DNS, etc. but you get the idea.)

Do either or both no. 2 and 3 sound like something we could pursue? Or do you have other ideas of how to accomplish these goals?

wsalesky commented 5 years ago

@nathangibson This may be a conversation to have with Dave as well in terms of long term viability. I have been experiencing a lot of instability with variouse eXist-db installs which have left me with a lot of questions about long term viability.

  1. I think investigating an option to output the site as static HTML sounds eminently reasonable, in addition I am very interested in the staticSearch for a different project I will be working on this year, so I can let you know about the feasibility of that as well. The redirects would have to be set up on the server, via jetty or apache httpd I would think. I can investigate what it would take to periodically out put a static version of the site as well as redirects.

  2. RE: Docker you can already do this with Docker. I can help you set it up in terms of having a Docker image for your project with your data and app and necessary libraries pre-loaded.

nathangibson commented 5 years ago

Thanks @wsalesky. I've put this on the docket for the next Syriaca meeting, not sure when that'll be.

  1. Great, it would be fantastic if you can investigate this. Since Biblia Arabica wants this, you can put in the time as part of that project. But before you start changing the Srophe app we should discuss with the other Srophe stakeholders as you said, and we will need to check the scale of the coding operation -- whether it is undertaken by BA alone or in partnership with others.
  2. I would love your help knowing how to do this with Docker. We can walk through it here or via Skype but more importantly, could you document it? You could do that using the Wiki page at Srophe perhaps? How do you want to proceed?
wsalesky commented 5 years ago

The next meeting is Monday, I can send you the time via email.

  1. I can add the Docker documentation to the Srophe app wiki, that makes the most sense.
nathangibson commented 5 years ago

Thanks, let me know when you've done that.

nathangibson commented 5 years ago

I mean the Docker documentation.

wsalesky commented 4 years ago

@nathangibson Here is the documentation for running the srophe app with docker and docker-compose: https://github.com/srophe/srophe/wiki/Deploying-with-Docker

@davidamichelson expressed an interest in the static HTML version of the record pages as well, and we have a partial solution developed for https://github.com/srophe/bethqatraye. I will go ahead and build it out so it is an XSLT that can be run on the github data which can then be mounted as a 'shadow site' as needed.

nathangibson commented 4 years ago

Checking up on open issues ... what's the status on this? I should also try out the Docker instructions :-)

wsalesky commented 4 years ago

@nathangibson The issue has stalled. I simply need to adapt the XSLT scripts to output a full HTML record, it should only take an hour or two, so I will try to fit it into my schedule for the coming week. Thanks for the reminder.