FoxxMD / laravel-elasticbeanstalk-queue-worker

MIT License
52 stars 13 forks source link

Deployment with Web & Worker Nodes [Clarification] #12

Closed MCKLtech closed 5 years ago

MCKLtech commented 5 years ago

I wanted to clarify the deployment procedure in the case of Web and Worker Tiers. Assuming I have the same Laravel codebase deployed on my Web and Worker Tiers, am I correct that only the worker tier(s) should have:

IS_WORKER = true

While all Web tier(s) should be:

IS_WORKER = false

Forgive my potential ignorance, but if the Web tier handles a request and pushes a job to a queue, how does the Worker tier know about it? I assume artisan on Supervisor is always listening to the queue and detects the new job being pushed by the Web tier?

FoxxMD commented 5 years ago

Your question is valid! I don't really explain this in the readme. Answering your last question first:

As you know AWS EB has two types of environment, worker and web.

Web environments are plain environments configured to handle web traffic, but with no mandatory assumptions about how the application will interact with the environment (other than to potentially accept HTTP requests at an endpoint).

Worker environments are configured with the mandatory assumptions that your application will consume an SQS queue and that it will do this by passively accepting jobs through HTTP requests from a daemon locally in the environment.


Queue workers in Laravel operate with different assumptions. A Laravel worker actively polls and consumes -- IE it does the requesting, rather than having requests sent to it -- jobs from a specified queue connection (and optionally a specified queue on that connection).

So, to summarize:


The deployment code in this repository helps fill the gap for running laravel workers in an EB environment by automating laravel's own flavor of queue consumption and worker management, supervisord. This allows you to use laravel's queue workers the way laravel intends so you don't need to be dependent on a EB worker's constraints.

The folder .ebextension in this repository is copied to your project. EB executes the files in this folder when it deploys an application. The files parse configuration for supervisord from environmental variables in the EB environment and then tell supervisord to run the workers that are generated.

So to answer your first question: The deployment code inside .ebextensions is run on every application deploy. The code doesn't know (or care) if the application is running in a Web or Worker environment. But you, as the deployer, probably want to differentiate between normal Web environments, not running workers, and the environments that will just run workers (or maybe you don't care as well).

When the deployment code is run the first thing it does is check if IS_WORKER=true. If the variable is present and is true then it continues the deployment (install supervisord, parse worker config, start supervisord). If the variable is not present or is not true then it exits immediately (workers are not created and run).

FoxxMD commented 5 years ago

@MCKLtech did that answer your question?