Run on branches

[craigmonson]

This will execute branch builds like they did in the make style stystem (see #4). Destroy will build the branches in reverse (a bug noted by mr. Joey in the make system).

This branch also renames the config struct from ColonizeConfig to just Config, and replaces all of that across the code.

Also added a simple test.sh to execute all our tests.

[jyore]

Clean is not working from branch or leaf level

$ cd /Users/jyore/Code/colonize-example-project 
$ colonize clean
Clean failed to run: read /Users/jyore/Code/colonize-example-project: is a directory

EDIT: This is actually happening for all commands

[craigmonson]

hmmm...

OOOOooohhh.... do you have build_order.txt in your path? I bet that's it.

Might need a better error message with that one.

[jyore]

Project structure:

.
├── LICENSE
├── README.md
├── app
│   ├── build_order.txt
│   ├── database
│   │   └── main.tf
│   ├── env
│   │   ├── dev.tfvars
│   │   ├── security_groups.tf
│   │   └── vpc.tf
│   ├── instances
│   │   ├── ami_map.tf.dev
│   │   └── main.tf
│   └── security_groups
│       ├── main.tf
│       └── only.tf.dev
├── build_order.txt
├── env
│   ├── dev.tfvars
│   ├── nonprod.tfvars
│   ├── provider.tf
│   └── remote_setup.sh
└── vpc
    ├── main.tf
    └── subnets.tf.nonprod

It happens at both Leaf and Branch level

EDIT: tree doesn't show hidden files, but the .colonize.yml file is indeed there

[craigmonson]

I'm not having the same problem. You sure the latest version is installed?

[jyore]

jyore:colonize/ (master) $ git fetch                                 [12:44:26]
jyore:colonize/ (master) $ git checkout run-on-branches              [12:44:35]
Switched to branch 'run-on-branches'
Your branch is up-to-date with 'origin/run-on-branches'.
jyore:colonize/ (run-on-branches) $ git pull                         [12:44:39]
Already up-to-date.
jyore:colonize/ (run-on-branches) $ go build
jyore:colonize-example-project/ (master✗) $ /Users/jyore/Code/gocode/src/github.com/craigmonson/colonize/colonize plan -e dev
Plan failed to run: read /Users/jyore/Code/colonize-example-project: is a directory

[jyore]

Ok, the problem was that my .colonize.yaml was missing the branch_order_file param, so It was considering everything a branch, which resulted in the odd behavior I was seeing. I added that param and everything is working now

[jyore]

I might have spoke too soon. Running into several things...

Nested Branches

It looks like it has to have the .colonize.yaml at the same directory path as build_order.txt.

In the example of nested branches:

- main-branch (.colonize.yaml is here too)
  - top-leaf
  - sub-branch
    - 1-leaf
    - 2-leaf
    - 3-leaf

you can only run from the top-level main branch. It will go into and run the leafs in the sub-branch, but you cannot execute commands from the sub-branch itself, or else you get an error like this:

Error loading config: No Terraform configuration files found in directory: /Users/jyore/Code/colonize-example-project/app

I think nested branches is a necessary part of this feature

Plan receives same plan for each branch

If I have 3 leafs in my branch and I run a plan. It reports the Running <leaf-name> correctly for each leaf name, however, the plan in each leaf is exactly the same as the first leaf. Interestingly, the _combined* files are created in each leaf, however, only the first one listed gets the terraform.tfplan file in its directory, so my guess is that it is executing the same tfplan (from first leaf) each time.

Leaf dependencies

Additionally, and this may be a separate feature issue itself, but I think we need some sort of a plan-apply command and/or a way to build dependency tree with the branch runs.

plan-apply: automatically (maybe a confirmation input and/or a flag to accept) run apply after you have planned (assuming no error) dependency tree: If possible, do not run plan/apply on dependent leafs until required changes are applied

Here is the scenario where 1 or both of these things would be handy: Assume you have a project where you have a leaf called security_groups where you define your security groups for your environment. Then you have another leaf that builds your EC2 instances, called webservers. You will need output from security_groups in order to correctly launch your webservers. You create a build_order.txt to make this a branch, where you have both leafs listed, so that it runs security_groups and then webservers.

Now, if you run a colonize plan -e dev to build out your dev environment, it will go through and run the plan on each leaf. The issue is, webservers cannot plan correctly, because the output from security_groups does not exist yet. So, you have to plan, apply, plan, and apply, in order to correctly build your webservers, which is the same as just running at the leaf level each time.

[craigmonson]

Latest commit should fix the problems with the branch running. (pewpewpew I hope)

[jyore]

Looks like the last commit fixes the "Nested Branches" and "Plan receives same plan for each branch" issues. The "leaf dependencies" I think would be better served as a future "enhancement" and I'll create a separate issue for it