This Composer package provides the starts for what you need to test and deploy a hosted site on something like Acquia.
If you are using the standard scripts that ship with this package, upgrading
should be easier. With each version upgrade, you should check your
.circleci/config.yml
file for changes against the config.yml
file here.
The easiest way to do this is to use a standardized diff tool and diff
.circleci/config.yml
with config.yml
. An example may look like:
diff -u --color .circleci/config.yml ./vendor/fourkitchens/pots/config.yml
Validate the changes and move over the ones you want to keep.
This package expects your project to have the following support scaffolding to run out of the box:
"scripts": {
"lint": [
"./node_modules/.bin/eslint ./",
"find web/modules/custom web/themes/custom \\( -iname '*.php' -o -iname '*.inc' -o -iname '*.module' -o -iname '*.install'-o -iname '*.theme' \\) '!' -path '*/node_modules/*' -print0 | xargs -0 -n1 -P8 php -l"
],
"code-sniff": [
"./vendor/bin/phpcs"
]
}
"scripts": {
"theme-build": "cd ./docroot/themes/custom/sdsu && npm run build"
}
Set up of the Circle tasks assumes you are doing this with a bot user. Please ensure that the bot user you are using has account on both the GitHub organization where your site lives and the hosting provider. You will need:
Install the package and copy over the template config.yml
composer require fourkitchens/pots
mkdir .circleci
cp vendor/fourkitchens/pots/config.yml .circleci/config.yml
The first pass will always fail. Move on to configuration.
There are some configuration steps that are shared between hosting environments. Make sure to check out the settings specific to the environment you are pushing to.
A Deploy Bot user will be needed. This tooling assumes Bender (the Four Kitchens deploy bot) as the default, but does not provide any credentials you need to set the tooling up. Ask a Web Chef about Bender, have the organization you are working with create a new bot, or create the new bot for them using the steps below. You will need to log in as this bot to do some of the configuration.
+
in
your email. See how to create task-specific email addressesAn SSH key is needed to be able to push commits from circleci to the hosting provider's git repository. In some instances, the same ssh key is also needed to be able to ssh into the hosting platform itself. Ask the client to generate an ssh key pair or generate and securely provide the ssh key pair to the client.
ssh-keygen -l -E md5 -f id_rsa
where idrsa is the filename of your key.You must have a GitHub token so the Deploy Bot can post comments back to commits or pull requests. This allows it to provide links and information about the environments it created.
If you skipped here, make sure you follow steps 1-6 of "Configure an SSH Key" to get to Circle's Project Settings"
Here you will configure the common environment variables. Hosting specific variables and steps will be provided farther down. The variables to configure are as follows.
GITHUB_TOKEN
A GitHub access token so the bot user can comment back on the
commit.GIT_EMAIL
An arbitrary email that will be used as the committer when
building the artifact.CANONICAL_ENV
The environment to get canonical database and files from.If you skipped here, make sure you follow steps 1-6 of "Configure an SSH Key" to get to Circle's Project Settings"
SITE_NAME
: The Pantheon site id used to run terminus commands.TERMINUS_TOKEN
: The Pantheon Machine Token used to operate terminus.For Pantheon hosted sites, we need an account for our Deploy Bot so that our deployments are linked to any one pantheon user.
Your bot will need an ssh key configured to be able to push commits to Pantheon. Use the public key of the pair generated in the "Configure an SSH Key" section.
Log out of your Pantheon account or use a separate browser instance to follow the instructions from Pantheon to Add Your SSH Key to Pantheon.
TERMINUS SITE ID
) of your Pantheon site into "Value".
Note: you can find the site id by looking at the site's dashboard in
Pantheon. If you click the "Visit Site" button on any environment the url is
in the format of
https://{{TERMINUS ENV}}-{{TERMINUS SITE}}.pantheonsite.io/
SITE_NAME
: The acquia site id used to run Acquia Cloud API commands.ACQUIA_REALM
: The Cloud API Realm. Usually "prod" or "devcloud".ACQUIA_REPO
: The Acquia git repo.
TODO: FILL OUT THIS BIT
Your bot will need an ssh key configured to be able to push commits to Acquia. Use the public key of the pair generated in the "Configure an SSH Key" section.
Log out of your Acquia account or use a separate browser instance to follow the instructions from Acquia to Add a public key to an Acquia profile.
There are many options you have, some shared, and some hosting specific.
Currently, you can configure your entire script by setting up some pipeline parameter defaults. These can be changed via API calls (advanced usage), but the defaults will be used when you are making pull requests or commits directly to the repository that houses your .circleci/config.yml.
In each one, change the value after the default:
key. For example, to change
the docroot across the whole project, change:
docroot:
description: "Where composer installs drupal."
default: "web"
type: string
to
docroot:
description: "Where composer installs drupal."
default: "docroot" #NOTE I CHANGED THIS
type: string
The following parameters exist:
This is a string that represents the version number for PHP version to use across the entire build.
If you are using Pantheon as a host-variant, this can only be a major.minor
version number such as "8.1"
because Pantheon Docker containers only allow
the two digit version.
On other hosting platforms, you can use a three digit version number
(major.minor.patch), such as "8.1.4"
if required, however 2 digits are
allowed.
This is a string that represents the timezone. This may be important so that jobs that do time calculations, print time stamps, or make commits are accurate. See a full list of timezone values to find an appropriate one for you to use.
This affects how the whole build behaves and what configurations are available to you. The current options are "pantheon", "acquia", and "general". All our workflows will provide you with "Artifact Build", committed the appropriate repository. This allows you to use Git history to follow a build back to the source commit we use in development.
Pantheon provides you a workflow that builds multidev environments, options around development branches, and a workflow to deploy code via approval through the CircleCI user interface. The deploy process provided by this setting creates an Artifact Build using Pantheon's Terminus with the aid of the Build Tools plugin. These changes are committed only to the Pantheon git repository and not to your "Source Repository", the one you create coding changes in.
Acquia provides you a very minimal workflow, however, it does provide acquia cli as part of the build. This allows you to use supporting deploy methods like our cloud hooks to streamline the deployment process. For multidev capabilities, it is recommended to use tugboat. The deploy process provided by these settings creates an Artifact Build using some standard git commands. These changes are committed only to the Acquia git repository and not to your "Source Repository", the one you create coding changes in.
General provides you with the barest of deployments, committing an artifact
build back to your source repository under a new branch that follows the naming
convention deploy-{BRANCHNAME}
. Example, if your branch is named test
, you
will have an artifact build committed to the deploy-test
branch.
This setting should be set to the folder where your Drupal installation resides.
This should mirror the setting in extra.drupal-scaffold.locations.web-root
.
This is typically either web
for Pantheon and docroot
for Acquia, but can
be any number of things like public_html
for generic hosts.
This setting moves where the built artifact should be. This is particularly
helpful when you want to pick and choose items from the build versus just
sending the whole artifact to your host. The ~/project
directory is always
committed to an "Artifact Build". Therefore, by changing this directory to
something like /tmp/project
, you can use rsync to move parts built by
the previous build script into ~/project
. By default, this value of this setting
is ~/project
, so that everything that was built during the "build" job is then
pushed to the host.
There are other configuration that cannot be affected by API calls and requires you to modify your config.yml directly if you wish to change them temporally or permanently.
There are the directories that you want to be copied in full from the build
portion of the workflow to the deploy portion of the workflow. You will
sometimes want to include other root directories that aren't included like
node_modules
, simplesaml
, or private
. We exclude most top level
directories because they are unneeded on the hosting system itself.
Environment variables can be configured using the CircleCI Environment Variables
interface, or you can set them directly in .circleci/config.yml
. Changing them
in the interface makes upgrading your .circleci/config.yml
less problematic,
however, it hides some of the toggles you may be using. Per best practices,
make sure any secret is configured in the CircleCI UI. We also include all
mandatory environment variables in this way as well.
GITHUB_TOKEN
: Mandatory A GitHub access token so the bot user can comment back on the
commit or PR, and remove unneeded multidevs.SITE_NAME
: The Pantheon or Acquia site id used to run terminus/acli
commands. Defaults to the GitHub repo name.GIT_EMAIL
: An arbitrary email that will be used as the committer when
building the artifact. Defaults to bender@fourkitchens.com
CANONICAL_ENV
: Environment to get canonical database and files from
Possible Values:
Default: "prod" on Acquia. "live" on Pantheon
SANITIZE_SCRIPT
: Script used to sanitize databases. Only used when
CANONICAL_ENV
is not dev. There is no default.SYNC_CONFIG
: The ability to turn configuration sync on or off. By default,
Yes if Any directory in the ./config directory (inclusive) contains
system.site.yml
. Ex: YES if ./config/system.site.yml
or
./config/default/system.site.yml
or ./config/sync/system.site.yml
exists. Possible values are YES
, or NO
.TERMINUS_TOKEN
: Mandatory The Pantheon machine token.CI_BUILD
: Build CI multidevs on every commit on Pantheon. This way you get
the ci-* environments. This may be useful for visual regression testing or
workflows without PRs. Defaults to NO
. Possible values are YES
and NO
.MAIN_BRANCH
: Define the main branch releases are cut from. Defaults to
main
if the branch exists, master
otherwise.DEVELOPMENT_BRANCH
: Define the development branch where active development
happens on GitHub. This branch is used most in gitflow development patterns.
Defaults to develop
.REBUILD_MULTIDEV_ENV_EVERY_PUSH
: Re-sync content for multidevs every time a
push is made to Pantheon. Defaults to NO
. Possible values are YES
and
NO
.REBUILD_DEVELOPMENT_ENV_EVERY_PUSH
: Re-sync content for the GitHub
development multidev on Pantheon every time a push is made to
DEVELOPMENT_BRANCH
branch. Defaults to NO
. Possible values are YES
and
NO
.DEVELOPMENT_ENV
: Define the name of the multidev used for the GitHub
development branch. Must follow the multidev naming conventions for Pantheon.
Defaults to github-dev
.ACQUIA_REPO
: Mandatory The address of the Acquia git repo. Example:
sitename@svn-21939.prod.hosting.acquia.com:sitename.git
.ACQUIA_REALM
: Mandatory The Acquia Cloud API Realm.
Usually "prod" or "devcloud".
See documentation.
Defaults to prod
.These environment variables are unconfigurable. They are set to help you with developing scripts on your own. Some contain the values of the pipeline parameters previously mentioned.
HOST_VARIANT
: Contains the value provided by the host-variant pipeline
parameter.DATE_TIMEZONE
: Contains the value provided by the tz pipeline parameter.DOCROOT
: Contains the value provided by the docroot pipeline parameter.TERM
: "xterm"There are 2 scripts that the default scripts require you to build so that it's reusable across the project. You must be able to run the following commands in your project.
composer run lint
composer run code-sniff
See the Dependencies section for examples.
Almost all actions provided by this framework are overloadable. By simply
providing a script in the correct location in .circleci/scripts
, you can make
this tooling do what ever steps you want to. We provide a default set of actions
in the scripts section of this repository which you can use to base your custom
scripts off of.
The scripts available for overload are as follows:
test/static
: Provides static code testing such as linting and code standards
validation. By default, this runs composer run lint; composer run code-sniff
build/php
: Provides functionality to build your application. By default,
this is simply optimizing a production ready dependency install.build/theme
: Provides functionality to build your custom themes. By default,
this runs npm run theme-build
and runs through $DOCROOT/themes/custom
and
performs a "cut" action on each .gitignore, so you can provide both a list of
items that should be ignored while coding and a list that's ignored when
building your artifact.deploy
: All the actions needed to push the Artifact build to the remote
hosting environment. On Pantheon, this includes creating and building the
multidev. On Acquia and General hosting, this just gets the Artifact code
to the repo it should live in.drush-commands
: Provides the basic deployment drush commands to run update
hooks, clear the cache, and sync configuration.drush-config-import
: Provides the commands for importing configuration. By
default, this provides drupal 8+ ready commands like drush cim
, but can be
swapped out for something like drush fra
if on drupal 7 or have a features
based site configuration setup.post-drush-commands
: Some sites require changes to external systems, such
as solr, third party cache, or want to run drush deploy:hook
to allow drush
to run HOOK_deploy_NAME()
as some post deploy cleanup code. See
Drush docs.deploy-to-test
: Provides the deployment logic to get changes to the
staging environment. By default, this provides the "deploy" services, syncs
live to test, and a standard deploy.deploy-to-live
: Provides the deployment logic to get changes to the
production environment. By default, this provides backup services, the
"deploy" services as well as a standard deploy.POTS contains steps and scripts to automatically look for security updates and install them into the different contributed projects.
This was build making use of CircleCI PipeLine parameters to trigger the CI job that run that scripts that checks and gets the secutity updates.
To make it trigger recluntly we have a couple of options:
To make the Automatic security updates works for our project, we will require to follow the next steps:
When continuous integration is configured on a project, CircleCI generates a set of SSH keys and automatically adds the private key to CircleCI and the public key to GitHub. This way CircleCI will have access to the repository in GitHub, but this access is just read-only. Originally for POTS this access is enough since it only needs reading the repository on GitHub and writing on the platform where the website is hosted (Acquia, Pantheon, Platform, etc).
With the arrival of AutoSec to POTS, this set of SSH keys is no longer enough, because AutoSec requires reading the repository, executing the security updates, and sending the changes to GitHub, therefore starting with AutoSec it is necessary to have read/write keys. CircleCI cannot generate keys with write access, therefore it is necessary to externally create a set of SSH keys and manually add them to both CircleCI and GitHub.
Project Settings -> SSH keys -> Additional SSH Keys
.Project Settings -> Deploy Keys
.AutoSec has been developed to use IA and obtain some functional tests of the packages it updates automatically. These steps are added to the PR documentation, thus providing important information to the developer performing the tests.
The AI implementation was carried out through the ChatGPT API, therefore, a key is required to be generated and added to CircleCI. It is important to say that the use of this key has a cost for each request, so the configuration of said key is totally optional, the system is made to check if a key exists, it asks ChatGPT for the steps and if it doesn't have the key configured, it doesn't do any request.
Project Settings -> SSH keys -> Environment Variables
.OPENAI_API_KEY
, and the value paste the hash code of the step 3.