bin/first-time-setup.sh
You can then follow the instructions in the following sections to run your playbooks:
Install ruby and python
If using [asdf], install the plugins as listed in .tool-versions
asdf install
pip install pipenv
NOTE You may encounter an error include the header <string.h> or explicitly provide a declaration for 'memcmp'
. Many thanks to folks here for solving this (https://github.com/openssl/openssl/issues/18720#issuecomment-1185940347). Instead run:
optflags=-Wno-error=implicit-function-declaration ASDF_RUBY_BUILD_VERSION=v20220630 asdf install
NOTE You may need to run asdf plugin-add python
brew install python
brew install rbenv
brew install pipenv
Note: These commands should be run for each shell you run
bin/setup
pipenv sync
pipenv shell
NOTE: The bin/setup
is very important since it adds a pre-commit hook to your environment that will prevent you from accidentally checking in unencrypted vault files.
Make sure docker is running before you run the following (from inside the pipenv shell
) to test the installation:
cd roles/common
pip3 install 'molecule-plugins[docker]'
molecule test
In all the steps below substitute your role name for your_new_role
Initialize the role with ansible-galaxy Run the following command from the root of this repo:
export your_new_role=<fill in the role name here>
cd roles
ansible-galaxy role init $your_new_role
cd ..
Set up to run from github actions vi .github/workflows/molecule_tests.yml
add for your role at the end matrix of the roles
- your_new_role
Setup the directory to run molecule
copy all molecule and lint files (note you need the .
in the command
below to get the hidden files)
cp roles/example/. $your_new_role
edit vi roles/$your_new_role/meta/main.yml
and add a description
edit vi roles/$your_new_role/molecule/default/converge.yml
name: Converge
with name: your_new_role
Test that your role is now working All tests should pass
cd roles/$your_new_role
molecule test
Push your branch and verify that CI runs and passes on GitHub Actions.
If the role is related to a new project, add group variables and inventory.
group_vars/your_new_project
directory and add files with the required variables. Usually this includes common.yml
for variables that apply in all environments, vault.yml
for secret values like passwords and keys, and one file per environment (generally at least production.yml
and staging.yml
).inventory/all_projects/your_new_project
file and list all VMs and other resources. Group them by environment - see any of the existing files for examples.inventory/by_environment/
directory. For example, add your_new_project_production
to inventory/by_environment/production
. Try to keep the lists alphabetized.You can run molecule test
from either the root directory or the role directory (for example roles/example)
If you are writing tests we have found it is easier to test just your examples by running from the role directory.
We also recommend instead of running just molecule test
which takes a very long time your run molecule converge
to build a docker container with your ansible playbook loaded. You can run converge and/or verify as many times as needed to get your playbook working.
molecule lint
molecule converge
molecule verify
If you are having issues with your tests passing and have run molecule converge
you can connect to the running container by running
molecule login
If you have a specific task that is not behaving, utilize the tests to run just that step. This is especially useful for long running molecule converge
You basically copy the failing task into the molecule/verify.yml and run verify over and over instead of needing to run the entire converge over and over. This makes debugging much faster and joyful!
If you need to ensure you're getting the newest docker image for your local test run you can do a dance like this to delete your ansible docker machines, volumes, and images:
cd to the role in question
% molecule destroy
% docker ps -qaf ancestor=quay.io/pulibrary/jammy-ansible:latest | xargs docker stop
% docker ps -qaf ancestor=quay.io/pulibrary/jammy-ansible:latest | xargs docker rm
% docker volume ls -qf dangling=true | xargs docker volume rm
% docker rmi quay.io/pulibrary/jammy-ansible
% molecule converge
If you need to run a playbook
ansible-playbook playbooks/example.yml
Running a playbook from an error or a specific task
ansible-playbook playbooks/example.yml --start-at-task="Task Name"
Check to make sure you're going to run on the correct host
$ ansible-playbook playbooks/figgy_production.yml --limit figgy3.princeton.edu --list-hosts
coordinate with operations to take the machine off load balancer
run playbook:
$ ansible-playbook playbooks/figgy_production.yml --limit figgy3.princeton.edu
capistrano deploy to the box that's off the load balancer.
Make sure you're depoying the already-deployed commit! Note the way to do this
may vary by project in figgy it's the BRANCH env var
$ bundle exec HOSTS=figgy3 BRANCH=commit_hash cap production deploy
SSH to the box that's off the load balancer and check that the index page still looks okay
$ curl localhost:80
A bunch of the page only shows up if you're logged in, but you can see the menus and stuff
Tell ops that one's done; switch the load balancer to the other box. Repeat the process.
After both boxes are provisioned, ops puts them back up on the load balancer
provision the workers:
$ ansible-playbook playbooks/figgy_production.yml --limit figgy_production_workers
Some roles don't have the right become
/ sudo settings. (Figgy doesn't have
this problem). the -b
flag to ansible-playbook will correct these.
how to do a dry run? it's unclear. documentation suggests it's --check
Finally, deploy from robots channel to get the new functionality on all the boxes at once
Currently there's no automation on firewall changes when the box you're provisioning needs to talk to the postgres or solr machines. See instructions for manual edits at:
Use ansible-vault edit
to make changes to the vault.yml
file, for example:
ansible-vault edit group_vars/bibdata/vault.yml
If you need to diff an ansible-vault file, run
git config --global diff.ansible-vault.textconv "ansible-vault view"
git config --local merge.ansible-vault.driver "./ansible-vault-merge %O %A %B %L %P"
git config --local merge.ansible-vault.name "Ansible Vault merge driver"
after which any git diff
command should decrypt your ansible-vault files.
If a file is not decrypting with git diff
you may need to add the file you're trying to diff to .gitattributes
.
More information about lastpass-cli can be found here: https://lastpass.github.io/lastpass-cli/lpass.1.html
brew install lastpass-cli
lpass login <email@email.com>
gem install lastpass-ansible
or asdf exec gem install lastpass-ansible
source princeton_ansible_env.sh
[WARNING]: Error in vault password file loading (default): Invalid vault password was provided from script
, it's possible you have vault passwords hanging around from previous projects, and they are overriding the lastpass password. If you no longer need those passwords, remove them. For example:rm -rf ~/.vault_pass.txt
rm -rf ~/.ansible-vaults
ERROR! Decryption failed (no vault secrets were found that could decrypt)
, you may still need to source the environment for your shell.
source princeton_ansible_env.sh
old_vault_password
server in lastpass. Replace the old vault password with the current ansible vault password. Add a note to include today's date.pwgen -s 48
to create a new password.ansible-vault rekey --ask-vault-password $(grep -Frl "\$ANSIBLE_VAULT;")
ansible-vault edit --ask-vault-password
on one of the files you changed (providing the new password), to validate that everything is as it should be.In a pipenv shell
pipenv sync
pipenv shell
Upgrade ansible
pipenv update ansible
If this fails you may need to
pipenv uninstall ansible
pipenv install ansible
Create the CI ansible environment
pipenv lock -r > requirements.txt
Create a PR and commit
Make recommended changes from dependabot PR run pipenv install -r requirements.txt
Check in changes to Pipfile.lock
Run the entire test suite locally
Re-run pipenv lock -r > requirements.txt