Massive playbook overhaul and cleanup

thoraxe commented 8 years ago

This is a fairly massive playbook overhaul/cleanup inspired by @detiber ha_support branch. Here are the salient points:

Switches to using a cloudformation template in order to make cleanup easier and to allow for easier use of multi-master (auto sets up load balancers and etc)
Combines cleanup into the run script and does away with the separate cleanup python script
Massive cleanup of run.py including new error handling, refactoring for readability and comments, and sanity checking
Massive cleanup and refactoring of the various demo-ansible playbooks including getting rid of the util_tasks folder, combination of certain playbooks and files, reordering, removal of deprecated tasks, naming of tasks, readability improvements, and more
Lots of confirmation required when running cleanup
a new debug mode for the run script that will launch a specific playbook with all options (for testing/evaluation purposes)

PLEASE PLEASE PLEASE run some tests before merging this. I am neither a Python expert nor an Ansible expert. But I was able to create and subsequently delete an environment without issue, so it appears to more or less be working.

I also tested that using either subscription management OR certificate-based repos I was able to get to the point of the openshift installation (demo-ansible) running and starting.

This was tested specifically against openshift-ansible-3.0.47-1

thoraxe commented 8 years ago

Just encountered an issue with single master. Please hold.

thoraxe commented 8 years ago

OK, new plan is to simply always use 3 masters. It's not that big a deal. Please hold.

thoraxe commented 8 years ago

Appear to be hitting https://bugzilla.redhat.com/show_bug.cgi?id=1317755 which is holding up testing.

thoraxe commented 8 years ago

I've now run this multiple times against multiple AMIs -- both the cloud-access 7.2 and the publicly available 7.2 AMIs.

I've run 4-5 tests with certificate-based installations.

I've run 1 test with RHSM.

The first run failed due to RHSM problems. The registration tasks have not changed, they have just been relocated, and the failure was outside of the task itself (eg: RHSM failed).

The scond run failed due to system instability. The failure occurred during the actual OpenShift installation (eg: openshift-ansible) and running a third time the installation successfully completed.

Once this installation is complete I will attempt to validate that the SDN/registry/router are working. At that point I would consider this merge-able.

Stay tuned.

thoraxe commented 8 years ago

I have been testing against openshift-ansible-3.0.47-1 (updated original PR comment).

This is good to test for others now. It seems to work under just about all circumstances. There are occasional weird issues with docker-storage-setup / docker daemon during initial installation but these appear to be related to AWS and not anything with demo-ansible or openshift-ansible -- they're out of the control of this particular repo.

detiber commented 8 years ago

@thoraxe I suspect that some of those issues may be docker-1.8 related, I ran into quite a bit of performance issues with docker-1.8 that all went away when I upgraded to docker-1.9

thoraxe commented 8 years ago

But Docker 1.9 isn't shipping with RHEL yet, so that's not relevant :)

detiber commented 8 years ago

closed in favor of #102

2015-Middleware-Keynote / demo-ansible

Massive playbook overhaul and cleanup #100