Add support for multiple OJS versions

[cormier]

First of all, thank you for your great work!

I tried the image and saw that it was not building the OJS version that I needed to test, so I parametrized the Dockerfile to make it build my version (https://github.com/lucasdiedrich/ojs/pull/32).

That works for now, but I believe that it would be easier to support different OJS versions with an an organised set of Dockerfiles.

Would you be interested in a PR that enables:

1. Supporting multiple OJS versions and structuring this repo by version and flavour. We could take example from the php docker project and use something like:

/ 3.1.2 / php5 / apache / alpine 

If someone prefers to use nginx and would like to contribute a PR for this it could be accepted in: 3.1.2 / nginx / alpine. Same thing if we decide to use debian stretch down later on.

I believe that this could help with some of the issues that were discussed in https://github.com/lucasdiedrich/ojs/issues/13. By having multiple Dockerfile for multiple cases we would be able to separate upgrading the OJS version from testing issues related to a php upgrade, for example.

Thanks again for the great work!

[lucasdiedrich]

Hi @cormier , using debian stretch inside an container env is not what we search for a lightweight container. Using multi tag container is a nice feature but for now we're focusing on the recommendations from PKP.

There is no need to support nginx because the user wants to use it, we need to focus on delivering and stable container to everyone can use for now. Once we automate the build tasks we can thing on creating new container tags.

Thanks for you help @cormier . Marking @marcbria to lets his thoughts on this.

[cormier]

Hi @lucasdiedrich,

Thank you for your quick reply. My proposal was not for an immediate implementation of nginx / stretch tags but rather for a way to organise the repo so that we can easily support these if (and only if) needs arise.

But you are right, maybe I am anticipating for needs that don't exist.

You mentioned automated build tasks. How do you see this ? Would you have something like:

1. One Dockerfile per version and sh scripts to generate those dockerfiles
2. A parametrised Dockerfile and CI build targets to generate the images
3. Something else ?

I could give some help moving towards that

Cheers

[marcbria]

Hi @cormier I'm happy somebody else finds this useful and likes to give feedback to improve.

Comments about your comment:

I tried the image and saw that it was not building the OJS version that I needed to test, so I parametrized the Dockerfile to make it build my version (#32).

Plase @lucasdiedrich correct me I'm you don't agree, but the plan is, when we feel work is good enough (and I'm quite close to this feeling :-D) the idea is creating labels to follow pkp's naming. With those labels, you won't need to parametrize the Dockerfile... you can refer the version you need in your docker-compose.

But if ARGs play well with PWD (will be a nice present to PKP if you can run demos on there) and we keep it by default to the last stable version I don't see any problem in going with this and it will be more flexible.

Once we automate the build tasks we can thing on creating new container tags.

Although I like a lot the idea of extra tags for nginx/stretch I'm very much agree with @lucasdiedrich on this. Baby steps and consolidate work before adding new features.

I love to see one day an official ojs image like described here: https://docs.docker.com/docker-hub/official_images/

But maybe this is to complex and we can carry on with something less restrictive.

At the end, the weight of the work is over lucas' shoulders and more tags will mean more testing because nobody is going to create CI builds right now... unless @cormier you like to join the group and assume this part. :-)

You mentioned automated build tasks. How do you see this?

We didn't talk much about it. @cormier suggest two ways as follows:

a) One Dockerfile per version and sh scripts to generate those dockerfiles. b) A parametrised Dockerfile and CI build targets to generate the images

Although I'm far from been a docker expert, second looks the right one... but I confess I can't see the lights and the shadows of each way and estimate the work it means.

Which do you prefer and why?

[lucasdiedrich]

@marcbria , totally agree with all your points, but i do think going for an CI could help on building targets but we can do that after we migrate the project to pkp branding (this naturallly will happen some time). The usage of the extra TAG could help the deveopment team to create an build for testing faster, and will be able to parametrized if some day we use the CI.

@cormier , i will be more than happy with you join on this project with us.

[cormier]

Hi @marcbria, @lucasdiedrich,

Thank you for your replies.

Once we automate the build tasks we can thing on creating new container tags.

Although I like a lot the idea of extra tags for nginx/stretch I'm very much agree with @lucasdiedrich > on this. Baby steps and consolidate work before adding new features.

Agreed. Let's not do these for the moment being.

I love to see one day an official ojs image like described here: https://docs.docker.com/docker-hub/> official_images/

But maybe this is to complex and we can carry on with something less restrictive.

It is indeed a lot of work -- and then they would have to accept the image, but they do show best practices that could be useful for these project even if it does not become an official image.

a) One Dockerfile per version and sh scripts to generate those dockerfiles. b) A parametrised Dockerfile and CI build targets to generate the images

Although I'm far from been a docker expert, second looks the right one... but I confess I can't see the lights and the shadows of each way and estimate the work it means.

Which do you prefer and why?

I looked at a few docker image projects and it seems like they going for the first approach. They have:

• A directory per version / variant
• Dockerfile templates for each variant
• An update.sh script to generate Dockerfiles per version / variant and update the CI script
• One sh script to generate tags for the images

So I would be in favour of the first approach because it seems to be the community approach and the one used in the hello world image

Before we venture into any of this, and to consolidate the work I thhink it would be useful to set up CI -- I will contribute work to help with this.

[marcbria]

This is great @cormier!! Thanks for your work! I'm really impressed.

But I'm unsure if we are moving in the right way. Sorry in advance if the questions are too obvious.

Summarizing, we want tags for:

• OJS versions (from 2.x to 3.x as in pkp)
• OS version (alpine, debian-slim...)
• Web server (apache, nginx...)
• PHP versions (7.2, 7.3...)

(Architecture is now out of our scope... )

In the moment we combine those tags, it will grow exponentially and will be a hell, isn't it?

So, some questions rises to me:

• Do we need an script and templates to do the job for us, or we are fine creating this manually?
• Do we need subfolders with the combinations? (ie: 3_1_2-0/alpine/apache/php72)
• How will be those dockerfiles called from the docker-compose?
• ...

Thanks again for your work @cormier and @lucasdiedrich

[cormier]

Thank you @marcbria for your remarks.

Summarizing, we want tags for:

• OJS versions (from 2.x to 3.x as in pkp)
• OS version (alpine, debian-slim...)
• Web server (apache, nginx...)
• PHP versions (7.2, 7.3...)

(Architecture is now out of our scope... )

In the moment we combine those tags, it will grow exponentially and will be a hell, isn't it?

You are right, this would be a lot of work. We could have tags for OJS versions, OS version, Web server, etc., but we don't have to, at least not for now. For this reason I don't think we should work on implementing all these use cases right away but wait until we need them or someone contributes a PR for them.

Do we need an script and templates to do the job for us, or we are fine creating this manually?

I think that right now we are fine creating these manually.

Do we need subfolders with the combinations?

Yes, but I would wait until someone contributes them to add them. In #34 I only needed one version so I added a OJS_3_2_1-0 directory. So if we merge this we would have an OJS version layer. The day we decide to move to php 7, for example, we could look into adding a php version layer on top of the OJS version layer.

How will be those dockerfiles called from the docker-compose?

Maybe our configurations differ, but I would not refer to directly to the Dockerfile and pull the image from docker hub instead.

Cheers

[marcbria]

Ok. More clear now. Thanks. :+1:

But let's go deeper in this before we start walking. ;-)

You are right, this would be a lot of work. We could have tags for OJS versions, OS version, Web server, etc., but we don't have to, at least not for now. For this reason I don't think we should work on implementing all these use cases right away but wait until we need them or someone contributes a PR for them.

I full agree in a progressive growing, but may be it's a good idea to plan how we will grow. I mean, what about define our tagging name and folder structure? (1) And won't be nice talking about the core tags that we will always ensure? (2)

About (1) I took a brief look to the official images documentation and faqs as well as the php, drupal and python images.

Here you have some examples of how they structure their dockerfiles and tag:

• php/5.5/apache/ ----------------> php:5.5-fpm
• php/7.3/stretch/apache/ -------> php:7.3.4-apache-stretch
• php/7.3/alpine3.9/cli/ -----------> php:7.3-cli-alpine3.9
• python/3.7/alpine3.9/ -----------> python:3.7-alpine3.9
• python/2.7/stretch/slim/ --------> python:2.7:slim-stretch
• drupal/8.6/fpm-alpine/ ----------> drupal:8.6-fpm-alpine
• drupal/8.5/apache/ --------------> drupal:8.5-apache

Although looks like there is no canonical way to do this, I suggest the following folder structure (as well as tagging naming):

• ojs
  • versionNumber
  • operatingSystem(-versionNumber)(-variant)
    • webServer(-versionNumber)(-variant)
    • php(-versionNumber)(-variant)

Let me clarify that "variant" will be defined as follows:

• on operatingSystems it will refer things like "slim" version as in "debian-9.8-slim".
• on webSever it will refer the fast cgi variant (fpm) as in "apache-2.4-fpm"
• on php it will refer the client version (just in case we like to deliver with and without cli) as in "php-7.3-cli"

It will be much more clear with an example, for instance right now we have:

• master -----> ojs/3.1.1-4/alpine3.7/apache2.4/php5 ---> ojs:3.1.1-4
• php7-test --> ojs/3.2.1-0/alpine/apache2.4/php7.2 ---> ojs:3.2.1-0-php7.2 (Versions as I remember, so don't take them as the real ones)

Although something is still blur in my head (for instance, how are we going to automatically convert folders in dockerhub tags), at this point I'm asking myself... even we didn't cover all the combinations (nobody does) why not starting from now with this folder structure and tagging naming?

Finally, about "the core tags that we will always ensure" (2), may be it's a question to be discussed in a different post and include PKP team in the discussion?

BTW, let me a OT: today we had a PKP technical meeting and I'm happy to announce that in a couple of months we will have more support from PKP. Alec said he like to be involved in this, but he can't till june or july.

Sorry for the long post. I didn't know how to summarize. :-(

@lucasdiedrich your opinion is essential. What do you think about all this?

[lucasdiedrich]

Hello @marcbria, im a little busy this week so i was not able to look deep into this, quick question, your idea is to create an Branch/Tag for each possible combination, nice, but there will be a lot of TAGs, the thing is, we should make the tags that PKP will support, so this maybe should be asked over PKP Forum?

As i work with devops (more ops than dev) i really hate debian images, they`re unecessary huge, but this project is not mine, its ours, so what are the recomendations from PKP?

One thing that im looking more deeply is the needed to use an external CI, like @cormier sugested. Today we use the CI provided by DOCKERHUB, and if were going create TAG we maybe never need to use another one. Everything is already integrated with Dockerhub.

I'm going to study a little bit more, but i'm liking what the project is becoming, i'll look into the PR and other stuff as soon as possible, sorry to take so much time to give a feedback.

Thanks.

[cormier]

Hello @marcbria, @lucasdiedrich

master -----> ojs/3.1.1-4/alpine3.7/apache2.4/php5 ---> ojs:3.1.1-4 php7-test --> ojs/3.2.1-0/alpine/apache2.4/php7.2 ---> ojs:3.2.1-0-php7.2 (Versions as I remember, so don't take them as the real ones)

Ok, we can do that. I have updated the PR to have the directory structure reflect on your suggestions. Regarding the tags, should we have the base php tag point to php7.2 instead of php5.6, since php7.2 is the latest version ?

Although something is still blur in my head (for instance, how are we going to automatically convert folders in dockerhub tags), at this point I'm asking myself... even we didn't cover all the combinations (nobody does) why not starting from now with this folder structure and tagging naming?

We can set tagging rules in the docker hub continuous integration interface. Which brings me to @lucasdiedrich's comment:

One thing that im looking more deeply is the needed to use an external CI, like @cormier sugested. Today we use the CI provided by DOCKERHUB, and if were going create TAG we maybe never need to use another one. Everything is already integrated with Dockerhub.

You are right that we might now have the need to use anything else than Docker Hub's CI. After reading your comment I looked at how it works and realized that I overlooked the features of Docker Hub when I introduced Travis to #34. I thought that Travis could be useful to test the PRs before they are merged to master, but I see that we can do that with Docker Hub as well. I will remove the Travis integration from the PR.

[marcbria]

im a little busy this week so i was not able to look deep into this

@lucasdiedrich I'm also like a lot how the project is growing, but please advice us if this is taking you too much time. This is not a sprint, it's a long term run and your role here is essential.

your idea is to create an Branch/Tag for each possible combination, nice, but there will be a lot of TAGs,

Not exactly this. The idea is not creating each possible combination, it's to plan in case we need to cover them one day.

I will propose start with the two images we have and as soon as we got them, create an script to autobuild dockerfiles and folders to cover all OJS 3.x and the last stable 2.x (over the same LAMP).

Does it make sense?

the thing is, we should make the tags that PKP will support, so this maybe should be asked over PKP Forum?

Full agree. I just mailed Alec and Clinton to join this conversation here (where they have context) but if you prefer I can open a new thread in the PKP forum.

As i work with devops (more ops than dev) i really hate debian images, they`re unecessary huge, but this project is not mine, its ours, so what are the recomendations from PKP?

I agree. In the last PKP tech meeting Alec commented people likes lighter images. I the other side, for devs, it's good to have an image with some toolkit inside.

As far as I know alpine is one of the lighters and the image is 280.7 MB (that looks fine for me). Probably we can make it even better cleaning the container removing nodejs and some other tools we just need for the installation? Anyway, it's a conversation we can have in future.

One thing that im looking more deeply is the needed to use an external CI, like @cormier sugested. Today we use the CI provided by DOCKERHUB, and if were going create TAG we maybe never need to use another one. Everything is already integrated with Dockerhub.

If we can accomplish the same test we have with travis with dockerhub it's fine for me. I mean, I usually prefer to keep thinks as simple as possible, but here I full trust in your both criteria.

I have updated the PR to have the directory structure reflect on your suggestions. Thanks a lot @cormier

Regarding the tags, should we have the base php tag point to php7.2 instead of php5.6, since php7.2 is the latest version?

Till 3.1.1-4 the recommended version is PHP >= 5.6 but the last release (3.2.1-0, that is also now the stable one) requires PHP >= 7.0.

So I think base need to be 7.2... at least until we discover what is happening with 7.3 (that it's supposed to be the last php stable version).

I thought that Travis could be useful to test the PRs before they are merged to master, but I see that we can do that with Docker Hub as well. I will remove the Travis integration from the PR.

Great. Thanks a lot for your PRs.

[asmecher]

Hi all! Sorry for being so slow to join the conversation -- I am currently running a prehistoric Debian and can't join the Docker party until I get around to an upgrade (likely with a new dev machine later this year). So I have literally zero experience with Docker at the moment but am excited to get into it when I have baseline support for it.

Rather than weigh into technical discussions about a tool I've never used, I'll point to a few policies that we're following that might help inform the best Docker approach...

• Vagrant image and branches. See https://github.com/pkp/vagrant/branches. In short, there are branches there named the same as the main branches of the OJS repository. This means when we create a new stable branch of OJS we also branch Vagrant, and as the toolset evolves the older Vagrant branches will continue to track. This is easier than trying to maintain a single Vagrant image that needs to understand e.g. how to install any version of OJS.
• Likewise, for plugins, we follow similar branch naming conventions (e.g. stable-3_1_2). Both of these decisions were made so that we can know with some certainty which branches of each repository are supposed to work with which branches of the applications, across potentially many repositories.

[marcbria]

Thanks for joining Alec.

As the main OJS developer/architect your opinion here is extremely welcomed and needed. ;-) About docker, I'm also new so don't worry we can learn together one from each others. Take your time, we are not in a hurry.

About the naming conventions...right we are not working on branches, we are working on tags instead and docker is sync with ojs tag-names, but:

• docker's is using dots instead of "_"
• docker avoid repeating "ojs" all the time.

Here you have the repo: https://github.com/lucasdiedrich/ojs

Let's see what @lucasdiedrich has to say about this, that is the one that is doing the hard work, but if we create new branches with the "stable" prefix and full pkp naming will be enough or you want us to rename the tags to follow pkp naming?

I'm asking because the idea with docker is going further than with vagrant and offer versions with diferent php versions, OS, webservers as is explained here.

Take care, m.