Closed SkipKeats closed 5 years ago
John, Hareesh, and Skip will work on diagramming refactoring ideas for Datagov-Deploy and compare.
Here are my thoughts on refactoring this repo:
In each repo structure accordingly using ansible role best practices so they can be pulled in using ansible-galaxy pull
to vendor in these broken out roles ^ (Also see https://github.com/ansibleplaybookbundle/ansible-playbook-bundle/issues/269 as a possible way to reuse our ansible work for future kubernetes deployment testing)
# Example for catalog ckan
defaults/
files/
handlers/
meta/main.yml # ansible-galaxy init file to pull in vendored roles needed for application deployment
tasks/
templates/
tests/test.yml # test playbook to run with circleci
vars/
vendor/ #role dependencies declared in meta/main.yml
.circle/config.yml # circleci v2.0 spec
.kitchen.yml # test-kitchen recipe to run ansible role in docker container using ansible-kitchen and kitchen-docker (used by circleci)
.kitchen.vm.yml # test-kitchen recipe to run ansible role in virtualbox using ansible-kitchen and kitchen-vagrant
Gemfile # for test-kitchen gems to simplfy ruby deployment
Dockerfile #
docker-compose.yml #docker-compose used for local testing of networked services (redis/mysql/postgres/etc) should use `build: .` to point to the Dockerfile and latest/standard services and port networking.
packer.json # packer template to run ansible-local provisioner for use on AWS auto-scaling-group/terraform work
.gitignore # .kitchen .vagrant Gemfile.lock
@JJediny: The above is what I was generally thinking; however, how would you deal with the current/past and future situations, i.e., vagrant/virtualbox and docker respectively? The files necessary for each scenario due not entirely match. And with vagrant, each Vagrantfile must be in its own directory, although the Ansible within it could call into another part of the 'tree'. If both Vagrant and Docker were to call to the same Ansible directory, we would need to differentiate yaml files that were specific to each type, e.g., v-datagov-web.yml or d-datagov-web.yml. At present, all of the 'application' yaml files sit at the top Ansible level. Thoughts?
@SkipKeats the current/past will need to be addressed with the use of variables
and when
statements in the roles. Also I'm not sure we want to use raw vagrantfiles as they are alot harder to customize, instead I think we should be using the test-kitchen framework to declare application specfic context. test-kitchen has bother a docker driver and vagrant driver and it makes it much easier to develop test cases in the future using the verifier
in test-kitchen to right acceptance test using inspec or the like. Thoughts
I will look into the test-kitchen framework.
https://github.com/neillturner/kitchen-ansible
Example for Docker driver
---
driver:
name: docker
use_sudo: false
privileged: true
#http_proxy: <%= ENV['http_proxy'] || nil %>
#https_proxy: <%= ENV['https_proxy'] || nil %>
transport:
name: sftp
max_ssh_sessions: 5
provisioner:
name: ansible_playbook
roles_path: ../
#requirements_path: requirements.yml
hosts: all
playbook: test.yml
ansible_verbose: true
ansible_verbosity: 4
ansible_extra_flags: <%= ENV['ANSIBLE_EXTRA_FLAGS'] %>
require_ansible_omnibus: false
ansible_connection: ssh
#http_proxy: <%= ENV['http_proxy'] || nil %>
#https_proxy: <%= ENV['https_proxy'] || nil %>
#Testing - Busser requires Chef to run test suites
require_chef_for_busser: false
require_ruby_for_busser: false
platforms:
- name: ubuntu-14.04
driver:
image: rndmh3ro/docker-ubuntu1404-ansible:latest
- name: ubuntu-16.04
driver:
image: rndmh3ro/docker-ubuntu1604-ansible:latest
- name: centos-7
driver:
image: rndmh3ro/docker-centos7-ansible:latest
suites:
- name: test
#verifier:
# name: inspec
# sudo: true
# inspec_tests:
# - https://github.com/dev-sec/linux-baseline
Thank you.
I have been looking at and experimenting with Chef Test-kitchen a bit, as well as looking at the assorted links above related to Ansible, Ansible Galaxy, etc. In my mind, I see a 3D cube of sorts per the refactoring, which consists of the following:
If I am missing an application, please let me know.
The current roles, as currently construed, would be split across these divisions. I am still trying to figure out how to deal with both Vagrant and Docker in the same primary repository, because I have not found any documentation that says it is allowable. I would think that we would need a primary file, probably a kitchen.yml, that would differentiate between current and future via switches of some form when run. I just not certain how to do that at present.
Thoughts would be appreciated.
John and Hareesh: FIrst draft of repositories per refactoring. Just want to make certain we are on the same page. Services and like not included yet and, per John's suggestion, will not go into the refactoring process in first go, e.g., kibana, etc. refactor_repositories_datagov-deploy.pdf
Per John's request, a SVG version of the refactor diagram is contained in the attached zip file. The primary file is DDRP.svg, but all of the icons are separate files. refactor_repositories_datagov-deploy.zip
All initial repositories are now created.
Per discussion with Hyon and team, datagov-deploy will be refactored into, essentially, the present/past (Vagrant/VIrtualBox) and the future (Docker). The how is under discussion.
Parent to #301, Parent to #302, Parent to #303, Parent to #304, Parent to #305, Parent to #306, Parent to #307, Parent to #308, Parent to #309, Parent to #310, Parent to #311, Parent to #312, Parent to #313, Parent to #314, Parent to #315