added docker files

[JasperLinmans]

What

Docker files required to build a docker which can automatically build and host one of the website. This will make building and changing the website locally easier. The docker is already live, and can be found here: doduo1.umcn.nl/webteam/website:latest. It automatically pulls the latest master branch from the website-content repository when starting a new docker instance. An additional argument can be given to automatically build and host a specific website. To do so: provide the name of the directory without the "website-" prefix.

Why

Solves GitHub issue #232

How

For instance, to automatically build and run the ai-for-health website: ./c-submit [put your c-submit arguments here] --interactive doduo1.umcn.nl/webteam/website:latest ai-for-health

Or for the diag website: ./c-submit [put your c-submit arguments here] --interactive doduo1.umcn.nl/webteam/website:latest diag

Go to port 8000 (which opens automatically) to view the website on your local machine when running the docker instance. The github repository is located in /home/user/source/website-content.

You can always build and host a specific website manually, by leaving out the additional argument in the c-submit command.

Example

A more detailed c-submit line, building the ai-for-health website as an example: ~/c-submit --priority=high --require-mem=4G --gpu-count=0 --require-cpus=1 --interactive [username] [ticket-number] 4 doduo1.umcn.nl/webteam/website:latest ai-for-health

[martvanrijthoven]

This looks very nice, very cool that it is compatible with the cluster!

Only one thing that we maybe can add, is a similar dockerfile that does not rely on the cluster/doduo-registry Maybe just a ubuntu base docker. This way it is not dependent on our cluster such that the docker can also run on other machines (locally, aws, etc).

[JasperLinmans]

Ah of course, I agree. Let's keep this PR open, I will add that as well soon.

[JasperLinmans]

Updated the docker files: it is now stand-alone, independent from the DIAG cluster. However, it is still compatible with the DIAG cluster: the image is stored in our image registry (webteam/website:latest) and running it will automatically expose the necessary ports.

Remaining TODOs:

• The images are not automatically copied, adding this would improve developing capabilities.
• The publication parse script is run, but the publications tab on the Diag website is not yet working. Let's discuss these remaining TODOs during our meeting.

[JasperLinmans]

Images are now copied automatically. Also, a local copy of the required bibfiles are included to be able to run parse_publications.sh.

Remaining TODOs:

• On our cluster, the website is pretty slow. I need to verify if this works better when running the docker image locally. If not, then this needs to be improved in order to be workable.
• Publications should be parsed correctly with the latest commit in this PR, but the publications page on the DIAG website is not working.

[JasperLinmans]

• Make sure to solve speed issues.
• Make sure, when you develop locally, running the docker on your local machine to reach the files on your local machine. This way you make sure changes are saved on your local machine and not lost when stopping the running docker image.
• Check what goes wrong in parse_publications.sh (right now it's running too fast, so something most go wrong there). Probably not running parse_publications.sh, only the python call in the end solves this.

[JasperLinmans]

• Checkout a python ubuntu version instead of the vanilla ubuntu docker image as starting point.
• Alternatively: remove the python3 install and related package install. Instead use a virtual env within the docker to install all necessary packages.

[martvanrijthoven]

For the alternative solution you can check this dockerfile (Lines 1-18 are relevant)

[JasperLinmans]

Updates since last time:

• Solved inefficiency issues.
• Solved publication parsing issues.
• Implemented local development.
• Solved ubuntu base image issues.

The current development docker is a very light-weight docker, hosting a local version of the GitHub repository.

Remaining TODO's:

• During next meeting: test docker on someone else's pc.

[JasperLinmans]

TODOs:

• Check on one other pc: building and hosting.
• Make it work with the cluster again, this will allow removing the develop branch (GitHub issue id here). To do so: make it compatible by cloning the master branch from GitHub, and afterwards checking out into a variable branch name to be able to host this branch.
• Final thing to fix: make the repository of bibfiles non-private, and just clone the recent version (updating the bib files accordingly).

[JasperLinmans]

• Implemented automatic cloning of our new public Literature repository.
• Implemented two different lightweight and easy to build dockers: one for local development and one for remote demo'ing purposes.
• The remote docker image is now fully compatible with our diag cluster and can host any branch specified through command line arguments.

This concludes all current feature requests. Let's discuss and demo this during our next meeting.