With Craft's job queue you can run heavy tasks in the background. Unfortunately, this is not entirely true. When runQueueAutomatically => true
(default), the job queue is handled by a ajax (FPM) call.
With many jobs in the queue and limited PHP-FPM processes this can break your site. This plugin replaces Craft's default queue handler and moves queue execution to a non-blocking background process. The command craft queue/run
gets executed right after you push a job to the queue.
Here you can find the initial discussion I started at craftcms/cms
.
Development happens in my free time, but also during working hours. Thanks fortrabbit.com!
This package is Treeware. If you use it in production, then we ask that you buy the world a tree to thank me for my work. By contributing to the Treeware forest you’ll be creating employment for local families and restoring wildlife habitats.
ostark/craft-async-queue:1.3.*
)cd your/craft-project
composer require ostark/craft-async-queue
php craft install/plugin async-queue
If you run into Composer version conflicts:
composer config platform --unset
composer update
php craft migrate/all
composer require ostark/craft-async-queue
php craft install/plugin async-queue
The plugin uses symfony/process to execute the php
binary. Usually the binary is located in /usr/bin/
, but other common locations are auto detected as well. With the ENV var PHP_BINARY
you can explicitly set the path, e.g. in your .env file like this:
PHP_BINARY="/usr/local/Cellar/php71/7.1.0_11/bin/php"
By default 1
background process handles the queue. With the ASYNC_QUEUE_CONCURRENCY
ENV var you can modify this behaviour.
# No concurrency
ASYNC_QUEUE_CONCURRENCY=1
# Or max 5 background processes
ASYNC_QUEUE_CONCURRENCY=5
To disable the plugin in certain environments, like on Windows which is not supported yet, set the DISABLE_ASYNC_QUEUE
ENV var.
DISABLE_ASYNC_QUEUE=1
Note that the plugin will overwrite the runQueueAutomatically
setting with Craft CMS. No configurartion required.
Beside the test suite you can run from the command line with this shortcut: composer tests
, you can perform a test in the Craft CP.
Navigate to Utilities
> Async Queue Test
and hit the Run test
button.
The command that runs in the background is basically php craft queue/run
, however we add some linux specific syntax that executes the command in a non-blocking way.
By setting useDefaultDecoration
to false
you prevent this. You have also the ability to modify the command itself.
// Add handler
\yii\base\Event::on(
\ostark\AsyncQueue\QueueCommand::class,
\ostark\AsyncQueue\QueueCommand::EVENT_PREPARE_COMMAND,
function(\ostark\AsyncQueue\Events\QueueCommandEvent $event) {
$event->useDefaultDecoration = false;
$event->commandLine = "BEFORE {$event->commandLine} AFTER";
}
);
Empty queue (only php-fpm master is running)
$ ps auxf | grep php
root 2953 0.0 0.0 399552 13520 ? Ss 12:27 0:00 php-fpm: master process (/etc/php/fpm.conf)
New job pushed (php-fpm master + child + /usr/bin/php daemon started)
$ ps auxf | grep php
root 2953 0.0 0.0 399552 13520 ? Ss 12:27 0:00 php-fpm: master process (/etc/php/fpm.conf)
app 3031 2.2 0.2 718520 45992 ? S 12:31 0:00 \_ php-fpm: pool www
app 3033 1.2 0.2 280936 32808 ? S 12:31 0:00 /usr/bin/php craft queue/run
app 3034 0.0 0.0 4460 784 ? S 12:31 0:00 \_ sh -c /usr/bin/php craft queue/exec "1234" "0" "1"
app 3035 1.2 0.2 280928 32280 ? S 12:31 0:00 \_ /usr/bin/php craft queue/exec 1234 0 1