magemojo / m2-ce-cron

Magento 2 cron project to fix bugs and common cron issues.
https://magemojo.com
MIT License
165 stars 45 forks source link

PTAL: Improved Cluster support (#87) #119

Closed JesKingDev closed 2 years ago

JesKingDev commented 3 years ago

Related Issue

https://github.com/magemojo/m2-ce-cron/issues/87

Overview

In scaled/clustered environments, there may be multiple app servers called upon to execute cron jobs. The cluster manager will use its own method to determine the appropriate server (via server load determination, configuration, or a service like Apache Zookeeper). In those instances (including for Magento Commerce Cloud, platform.sh, etc.) the actual cron:run command will be run on the server desired to be the cron execution server.

This PR introduces support for these environments with a new cluster_support configuration parameter and an execution_host column in the cron schedule table. The cron job PID is now paired with the hostname to ensure that the current server only checks for its own PIDs. Previous PRs already introduced the idea of cron PID mutiny, this takes it a step further to acknowledge that other servers could be actively running/finishing jobs at the same time.

In environments like AWS Elastic Beanstalk clusters and the like, the leader is somewhat stable and only changes occasionally or during deployments, etc. However, in Magento Commerce Cloud, the mechanism could potentially execute cron:run on a different server each time. With the more frequent switching back and forth, it's critical to detect and stop the execution loop.

NewRelic support is also improved. Currently, NR sees that the service is running for a long period of time and will attempt to collect all of the trace data to help troubleshoot a problematic transaction. This PR introduces reporting individual transactions for each loop of the service. This keeps transaction traces down since the loops will be short and sweet.

I also linted the project against the Magento coding standards, which resulted in a number of spacing corrections.

Testing

This was tested locally using Warden on Magento 2.3.7 with PHP 7.3.

JesKingDev commented 3 years ago

We've been running this in our MEC Pro env for over a week and are pushing to our production environment tonight. I will update this PR as we identify any issues with the implementation.

JesKingDev commented 3 years ago

Checking in - this is still running great in MEC Cloud.