Ansible-based infrastructure-as-code for hosting and deploying Magento-based shops.

Infrastructure Provisioning

CI/CD Automation

This code handles infrastucture provisioning, maintenance and deployment only. This repository does not contain any CI/CD automation for building, testing or running the provisioning itself. We do that internally with reusable Jenkins pipelines (as libraries) - as this setup is very specific to our workflow we are not planning to open-source it in the near future.

MageSuite ecosystem

Altough it's possible to use this code to host any Magento-based shop, some features are only avaialble (and tailored for) MageSuite ecosystem.

Cloud infrastructre

Currently the full feature set is only available when using AWS as the cloud platform, however, it's possible (and tested) to use this code to provision any CentOS 7 VM including local development environment.

PS There's also an experimental docker container which can be fully provisioned by this code and run a full CentOS installation like a VM. This is intended to be used only for local development though and not yet supported.

Feature highlights

• Out-of-the-box autoscaling
• Zero-downtime deployments¹
  • Nodes are warmed up when new code is being deployed before being attached to the varnish load-balancer to avoid cache generation penalty after traffic hit
• Varnish caching tailored for Magento with multiple configurable custom features
• Built-in support for elasticsearch / ElasticSuite
• Redis Magento cache backend with
  • Separate redis instance for session storage
• Customizable multi-website / multi-shop deployments
• Letsencrypt certificates support
• Cloudfront CDN
  • Media stored and shared across nodes using AWS EFS or AWS S3 (and possible plain NFS soonish)
  • Seamless image optimization via Lambda@Edge (with webp support)
• Support for arbitrary file sharing across nodes via AWS EFS
• Per path pattern customizable PHP limits
• Customizable request throttling (rate-limiting)
• On-demand import instance
  • EC2 instance started for scheduled imports automatically and terminated once the import is finished
  • Allows to use a different instance type for heavy scheduled imports and perform them without disrupting the site performance for end-users
• Customizable rules for routing traffic to separate set of special purpose extra-instances
  • Routing rules based on user-agent and request path patterns
  • Useful for processing heavy workloads without disrupting shop performance (e.g. ERP Magento API integrations)
• Magento queue consumers support with RabbitMQ and supervisord
• Logs pushed to CloudWatch with slack notifications for critical errors
• VPC and SG-based network communication rules allowing internal service communication strictly on a need-to basis
• Language and country-based redirects
• On-demand PHP debugging on production without affecting the shop's performance or configuration
• Advanced JavaScript Bundling support (via Dockerized AMD module bundler / preloader - Baler)
• ... any many more!

¹ Needs at least 2 app nodes and requires certain MageSuite extensions. Also it might not be possible depending on changes in relation to previous release in the code being deployed (DB migrations, ...).

Initial open-source release

The code has been cleaned up and all commits squashed for the first public release.

The previous internal repository had 2000+ commits, 300+ merge requests created by 10+ contributors across 3+ years of development.

The current state is a result of hundreds of hours of meetings, debugging, brainstorms investigations and combined expertise of all our developers, PMs and clients.

A lot of the solutions are tailored for solving real-world problems and use-cases that came up when hosting and maintaining highly-customized Magento-based shops. The value of such experience-based solutions cannot be overestimated. It wouldn't be possible to come up and design all of these tiny details form scratch without having went through the all of problems first-hand.

Documentation

At the time of open source release most of the documentation was scattered across internal creativestyle services thus any leftovers have been removed to avoid confusion. New documentation will start appearing shortly as part of this repository succesively.

Utilities

Available CLI commands on EC2 nodes

mageops-env # prints mageops env vars

mageops-sys-status # prints general info about cpu, io, memory, running processes

mageops-consumers # prints ongoing magento queue consumer processes

magcd # `cd` into magento directory
mag # `cd` into magento directory and runs `bin/magento`
magcf # runs `bin/magento cache:flush`

Brought to life by