cron / hosting-dispatch missing?

[helmo]

The task queue is being handled OK here but what about the other queues?

In a regular install we have a crontab entry that runs 'hosting-dispatch' to go through all queue's.

Not a big pain for me at the moment though .. I just missed it while looking at hosting_distributions in a dev container.

[jfayad]

Hey @helmo can you provide more details about what you did ?

I can find no details about that in the regular aegir install documentations.

Moreover, I started the drush @hostmaster hosting-dispatch command and nothing happened (the last time executed did not get updated on the Web interface), yet when I ran drush @hostmaster hosting-dispatch -v I saw the "Last run" updated yet the command never return to the CLI... so not sure how to "integrate" that within a cron tab ?

[PaulLebmann]

Hey! As far as I understand it, hosting-dispatch is running the cron - job for the sites if you use the cron-queue module from inside hostmaster. It also touches the civicrm cron jobs if you use the Civicrm Cron queue module (which I do).

Maybe it was clearer in older documentation. At least I had to set it up on my old hostmaster instance. I just find hints to cronjobs in the troubleshooting section on http://docs.aegirproject.org/en/latest/install/#manual-installation for solaris and this discussion https://www.drupal.org/project/hostmaster/issues/2230287

I think a cronjob should be created which calls /usr/local/bin/drush '@hostmaster' hosting-dispatch. Since I do not have too many sites I will try to mount a script into /etc/cron.hourly with my docker-compose.yml file and see if it does the trick for me.

[PaulLebmann]

Well, putting it in cron.hourly didn't work because is run as root and root doesn't know the site aliases. I also tried mounting a crontab file for aegir in /var/spool/cron/crontabs/aegir. It seems hosting-dispatch didn't run. Maybe the dockerfile has to be modified.

[PaulLebmann]

So ... /var/spool/cron/crontabs/aegir also didn't work and to keep with the spirit of docker a separate container would be preferable. So for now I use with https://github.com/willfarrell/docker-crontab which is added in docker-compose.yml with:

crontab:
    build: ./crontab/
    restart: always
    depends_on:
       - hostmaster
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./crontab/config.json:/opt/crontab/config.json:rw

with the following config.json file:

[{
        "name":"hosting-dispatch",
        "comment":"run drush hosting-dispatch so that hostmaster manages cron jobs for the sites",
        "schedule":"* * * * *",
        "command":"/usr/local/bin/drush '@hostmaster' hosting-dispatch",
        "project":"hostmaster",
        "container":"hostmaster"
}]

which seems to work.

If you want to use this, you also have to put the docker-entrypoint, in addition to the Dockerfile, from willfarrell/docker-crontab in your ./crontab subdirectory of your aegir-docker project.

An additional advantage of this solution is that I can add another cronjob which manages the database backups directly in the database server container and don't need another container with a database client and cron for it.

[PaulLebmann]

Just to keep you and future me up to date. So far it works perfectly fine. I just switched from building the image myself to using the original image willfarrell/crontab directly and got rid of the additional docker-entrypoint file and Dockerfile.

One could add an example docker-compose.yml to manage the cronjobs or add it to docker-compose.yml, describe it and comment it out so that users have the option to use it. I think this is rather an documentation issue than an implementation problem. Since there are a lot of different solutions to run cron jobs in the container I choose this one after a bit of research but still rather randomly. I'm sure it would also work with a lot of the other options.

[doka]

I had a same issue on a Debian 9 (stretch) based container.

Workaround was:

• install cron, since the official debian image does not have the cron package
• add sudo rigths to the aegir user to enable restart of cron service as aegir
• restart cron after the contrab entry of hosting-dispatch has been added

All these can be added to Dockerfile, entrypoints and sudoer files.

[jonpugh]

This is great, thanks for the detailed steps @doka

If there are any more specifics to how you achieved success it would help!

[doka]

More details to the steps to get cron queues running.

TLDR: install cron package and restart the cron service via aegir user after Aegir hosting queue module has been enabled.

Detailed steps:

• Install cron package in Dockerfile, since the Docker images of Ubuntu 16.04 and Debian 9 (stretch) do not have it:

apt-get install -y -qq cron

• add sudo rights for aegir user to restart cron service via aegir sudoer file as follows

Defaults:aegir  !requiretty
aegir ALL=NOPASSWD: /usr/sbin/apache2ctl, /etc/init.d/cron

• restart cron service in docker-entrypoint.sh, just after hosting_civicrm_cron has been enabled

echo "ÆGIR | -------------------------"
echo "ÆGIR | Start cron service ... "
sudo /etc/init.d/cron restart
sudo /etc/init.d/cron status

You can see the whole magic working in my Aegir-Docker repo.