Build AMIs on the Util Slave

[jacderida]

Hi Stephen,

This extends the util slave for building AMIs. The jobs for doing this are defined in the jenkins-jobs repo (there will also be a PR there) and the docker_build jobs have been extended to trigger these jobs.

In terms of issues:

• This closes #112
• This closes #114
• This closes #119

Setup Instructions

• Create the environment with make env-jenkins-staging-aws
• SSH into the Bastion
• Activate the virtualenv with source ~/.venv/provisioning/bin/activate (this is a refactor)
• Provision the environment with make provision-jenkins-staging-aws-initial

• Login and configure the seed job with the following (note the references to my forks, which is why I'm not asking you to paste the usual file):

  
  pipelineJob('ami_build-safe_nd_slave') {
  parameters {
      stringParam('BRANCH', 'ami_build')
      stringParam(
          'REPO_URL',
          'https://github.com/jacderida/safe-build-infrastructure.git')
      stringParam('SAFE_IMAGE_TAG', '')
  }
  
  description('Creates a Docker slave AMI for safe-nd')
  
  definition {
      cpsScm {
          scm {
              git {
                  remote { url('https://github.com/jacderida/jenkins-jobs.git') }
                  branches('ami_build')
                  scriptPath('ami_build-safe_nd_slave/Jenkinsfile')
                  extensions { }
              }
          }
      }
  }
  }

pipelineJob('docker_build-safe_nd_build_container') { parameters { stringParam('BRANCH', 'master') stringParam( 'REPO_URL', 'https://github.com/maidsafe/safe-nd.git') }

description('Builds and pushes the container for safe-nd')

definition {
    cpsScm {
        scm {
            git {
                remote { url('https://github.com/jacderida/jenkins-jobs.git') }
                branches('ami_build')
                scriptPath('docker_build-safe_nd_build_container/Jenkinsfile')
                extensions { }
            }
        }
    }
}

}

pipelineJob('docker_build-safe_cli_build_container') { parameters { stringParam('BRANCH', 'master') stringParam( 'REPO_URL', 'https://github.com/maidsafe/safe-cli.git') }

description('Builds and pushes the container for safe-cli')

definition {
    cpsScm {
        scm {
            git {
                remote { url('https://github.com/jacderida/jenkins-jobs.git') }
                branches('ami_build')
                scriptPath('docker_build-safe_cli_build_container/Jenkinsfile')
                extensions { }
            }
        }
    }
}

}

pipelineJob('ami_build-safe_cli_slave') { parameters { stringParam('BRANCH', 'ami_build') stringParam( 'REPO_URL', 'https://github.com/jacderida/safe-build-infrastructure.git') stringParam('SAFE_IMAGE_TAG', '') }

description('Creates a Docker slave AMI for safe-cli')

definition {
    cpsScm {
        scm {
            git {
                remote { url('https://github.com/jacderida/jenkins-jobs.git') }
                branches('ami_build')
                scriptPath('ami_build-safe_cli_slave/Jenkinsfile')
                extensions { }
            }
        }
    }
}

}

## Test Instructions

* Start both AMI build jobs one after the other, as quickly as possible:
  - For `ami_build-safe_cli_slave` use `0.0.1` for the `SAFE_IMAGE_TAG` parameter
  - For `ami_build-safe_nd_slave` use `0.1.0` for the `SAFE_IMAGE_TAG` parameter
* Wait for both of those jobs to complete
* Use the AWS GUI to spin up instances of the images produced by those jobs
* SSH to each instance and run `docker images` to make sure they have the correct containers pulled in
* Use the AWS GUI to deregister both of the images
* Kick off `docker_build-safe_nd_build_container`
* Wait for this job and the downstream `ami_build-safe_nd_slave` to complete

## Acceptance Criteria

* The environment should be created without errors (due to additional security group rules)
* The environment should be provisioned without errors (due to new variables supplied)
* Both concurrent AMI jobs should run successfully
* The slave images produced should have the correct containers pulled on them (this tests that Ansible targets the correct machine when more than 1 job is running)
* The docker build job should run successfully
* The docker build job should trigger the AMI build job
* The AMI build job should run successfully

Cheers,

Chris

[S-Coyle]

Acceptance criteria:

• [x] The environment should be created without errors (due to additional security group rules)
• [x] The environment should be provisioned without errors (due to new variables supplied)
• [x] Both concurrent AMI jobs should run successfully
• [x] The slave images produced should have the correct containers pulled on them (this tests that Ansible targets the correct machine when more than 1 job is running)
• [x] The docker build job should run successfully
• [x] The docker build job should trigger the AMI build job
• [x] The AMI build job should run successfully - https://jenkins-staging.maidsafe.net/blue/organizations/jenkins/ami_build-safe_nd_slave/detail/ami_build-safe_nd_slave/2/pipeline/