devops-experiments

Table of Contents

1. Summary of Task
2. Working Deployment
3. Setup Notes
   1. Steps Followed
   2. Ansible Tower
   3. EC2 with Vagrant
   4. Run Ansible Playbook to setup Dev Environment
   5. Troubleshooting
   6. References

1. You can make any assumptions and be as innovative and creative as possible in your usage of tools for DevOps tool-chain
2. You are expected to implement a CD pipeline with no use of shell scripts
3. Check-in the complete project (cookbooks, manifests, Jenkins build definitions etc.) into a GitHub account and send us the repository location
4. Use the spring application https://github.com/spring-projects/spring-petclinic/ as source for the CI And CD implementations
5. Feel free to use AWS and share the working installation URL also.
6. Recommended tool for AWS : Vagrant

1. The web application is deployed here. See image.
2. The Jenkins CI is located here.

The flow is as follows:

1. The job is set to poll the GitHub repository once every day.
2. Upon detecting changes it executes the ./mvnw command (as documented).
3. The JUnit test results get parsed as a post-build step.
4. Upon successful build, the WAR is deployed to Tomcat (using Jenkins Deploy Plugin).

Here is where some key Ansible-related files are located:

ansible-tower
├── ansible.cfg   - ansible configuration
├── hosts         - hosts file containing webservers group. Place EC2 hostname here.
├── playbook.yml  - main playbook that configures dev environment.
├── roles         - the main playbook executes different roles. 
│   ├── apache
│   ├── geerlingguy.java
│   ├── geerlingguy.jenkins
│   ├── jenkinsjobs
│   │   └── templates
│   │       ├── config.xml - jenkins job configuration XML.
│   ├── mysql
│   ├── tomcat    - this role takes care of tomcat configuration and startup.
│   └── tools
│       └── tasks
│           └── main.yml
└── Vagrantfile   - In order to start an Ansible Tower Instance.

1. Of Puppet, Chef and Ansible, I chose Ansible.
   1. Ideally I would have tried out all 3 tools and made an educated choice.
   2. But in the given time for the assignment, the few reviews I read seemed to indicate similar attributes for all 3 tools. Thus my choice.
   3. An additional weighting factor in favor of Ansible was that it seems easy to configure vagrant with an Ansible tower.
   4. I read that Puppet Master does not work on Windows. Given that my primary machine at home is Windows I discarded Puppet.
2. I chose geerlingguy/jenkins for the relative ease of configuring Jenkins as an Ansible role.
3. I set up Jenkins on the same machine as the web server.

cd ansible-tower
vagrant up
vagrant ssh

1. The vagrant ssh command gives you credentials to Ansible Tower GUI as well as SSHing into the Tower Instance.
   • I did not have to use the GUI at all beyond initial exploration.
2. Note: On my machine, vagrant up throws the error described here. I did not find time to solve this.
3. My experiments were conducted with Ansible Basic Tower. The Enterprise tower license required a delay of 1 business day.
4. I have used Vagrant to bring up Ansible Tower as per the Ansible docs.
5. The first vagrant up caused some issues that I solved (see troubleshooting).
6. See image of Ansible Dashboard.

• m3.medium
• Region: Singapore
• AMI: ami-e90dc68a
• Vagrant Box Used: lattice/ubuntu-trusty-x64

See image of successful launch.

Details on Connecting to Instance:

1. Make sure vagrant-aws plugin is installed.
2. Place the devops.pem at the top of the repository.
3. cd ec2-i-f9eaa477
4. vagrant up --provider=aws
5. (Alternatively you can also SSH in)
6. During my attempt, I encountered and solved several issues. (See troubleshooting).

2. devops.pem should be placed at top level of repo.
3. Permissions set to 0400 otherwise SSH will throw errors.
4. ansible-tower/hosts should contain the public DNS of the EC2 instance(s) you want to configure.
   4. e.g. ec2-12-34-56-78.ap-southeast-1.compute.amazonaws.com
5. The Ansible Role geerlingguy/jenkins needs installation as mentioned here. Run the command:
   5. cd ansible-tower
   6. ansible-galaxy install geerlingguy.jenkins -p ./roles/ (this will download the roles)
   7. Verify that the roles subdirectory now has

      roles/geerlingguy.java/
      roles/geerlingguy.jenkins/

Now run:

ansible-playbook playbook.yml

Note: the first run will ask you if you want to continue connecting. See image. I have kept StrictHostKeyChecking enabled even though I have the option to disable that and make it truly hands-free. This is for security.

Customization

The following defaults are stored in the roles/tomcat/defaults/main.yml playbook.

# tomcat download URL. override this as well as tomcat_filename, to download a different tomcat version.
tomcat_url: http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.61/bin/apache-tomcat-7.0.61.tar.gz
# the filename of the file downloaded from above URL.
tomcat_filename: apache-tomcat-7.0.61
# temporary download destination.
tomcat_download_dest: /tmp/
# install location of tomcat.
tomcat_install_dest: /usr/share/
# tomcat ports
tomcat_http_port: 8080
tomcat_https_port: 8443
# tomcat user credentials
tomcat_admin_username: admin
tomcat_admin_password: admin

One can override these variables as per the Playbook docs to override these default values.

E.g. ansible-tower/playbook.yml overrides the Jenkins Port default (as decided by geerlingguy/jenkins):

  vars:
    jenkins_http_port: 8095

• Deploying on EC2 with Vagrant
• Configuring Ansible Tower with Vagrant
• Search for Vagrant Boxes
• Puppet Pre-install
• Ansible AWS Guide
• Configuring vagrant-aws
• Connecting to AWS Instances from Windows
• Maven Wrapper
• Ansible and Jenkins
• Ansible and Tomcat
• Tomcat: 1
• Jenkins Deploy Plugin
• Jenkins Config and Deployment
• geerlingguy/jenkins
• other: 1, 2, 3, 4, 5, 6, 7

vagrant console error

The installation process had installed VirtualBox on my machine automatically, and created a VM. So I opened VirtualBox and tried to start the just-created VM. This showed me a more helpful error:

VirtualBox Error

Solution

1. Went to BIOS and enabled VT-x. (Lenovo-specific option: Configuration > Virtualization > Enabled).
2. Restarted Machine and VM.
3. This time the Ansible Tower VM came up properly. :+1:

SSH to EC2 instance: PEM file permissions cannot be changed by cygwin

Before Connecting to the EC2 instance using the downloaded PEM File, I would need to change the file's permissions to 0400.

However, on my Windows + Cygwin laptop, changing the permissions did not appear to work at all.

After tearing my hair out some, I figured out how to successfully change the permissions and connect to the machine. :+1: See the problem screenshots and solution in this comment.

Errors while deploying and connecting to Vagrant Box with EC2

I encountered multiple issues while exploring deployment of a Vagrant box to the EC2 Image. See problems and solutions here and here.

Error configuring iptables

See this comment.