Can we run any cookbooks using terraform provisioner?

[only4pri]

Can we run any cookbooks using terraform provisioner?

I want to run a cookbook once a server is built. How can i achieve it ?

[edwardbartholomew]

Greetings @only4pri and thank you for your interest in kitchen-terraform! Bootstrapping a server with Chef is something that can happen naturally with Terraform (without need for kitchen-terraform) as I'll describe below. The great thing about using kitchen-terraform though is that you can easily run Inspec tests to verify everything together - that changes made in either Terraform or your cookbook, didn't break your overall module deployment.

My typical workflow would have the Terraform module bundle the cookbook and upload to a s3 bucket with a null_resource. Then create a template that can be called during server bootstrapping. In this example, I've named it mycookbook and it exists in a different repo than the Terraform module) . If this cookbook uses other cookbooks, then you'll need to use berks package in this below step rather than just zip it up.

Use AWS CLI to upload mycookbook cookbook:

resource "null_resource" "upload_cookbook" {
  depends_on = ["aws_s3_bucket.mycookbook_bootstrapping"]
  provisioner "local-exec" {
    command = <<EOF
cd /tmp;
rm -rf mycookbook mycookbook.zip;
git clone git@github.com:newcontext/mycookbook.git;
zip -X -r mycookbook.zip mycookbook -x \*.git\*
aws s3 cp mycookbook.zip s3://${aws_s3_bucket.mycookbook_bootstrapping.id}
EOF
  }
}

Create a bootstrapping script to run when server starts:

#!/usr/bin/env bash

# install prereqs
apt-get update
apt-get install -y awscli unzip

# install chefdk (use a more current version)
wget https://packages.chef.io/files/current/chefdk/3.0.29/ubuntu/16.04/chefdk_3.0.29-1_amd64.deb
dpkg -i chefdk_3.0.29-1_amd64.deb
# from /tmp, download wrapper cookbook and unzip it
cd /tmp
aws s3 cp --region ${aws_region} s3://${s3_bucket_name}/mycookbook.zip .
unzip mycookbook.zip
# need to have HOME set to avoid something like: https://github.com/ConnorAtherton/rb-readline/issues/8
export HOME=/tmp
# download dep cookbooks and run chef-zero to configure
berks vendor --berksfile mycookbook/Berksfile /tmp/cookbooks
echo '${chef_attributes}' > attributes.json
chef-client -j attributes.json -z --override-runlist recipe['mycookbook']

Create template for above script (create another one if you want attributes like in this example):

data "template_file" "script" {
  template = "${file("${path.module}/templates/userdata.sh.tpl")}"

  vars = {
    aws_region      = "${data.aws_region.current.name}"
    s3_bucket_name  = "${aws_s3_bucket.mycookbook.id}"
    chef_attributes = "${data.template_file.chef_attributes.rendered}"
  }
}

When creating the instance, set this template to run via user_data:

resource "aws_instance" "repo" {
  depends_on = ["null_resource.upload_keys"]

  ami                  = "${var.image_id}"
  instance_type        = "${var.instance_type}"
  user_data            = "${data.template_cloudinit_config.config.rendered}"

There's multiple ways but that's one pattern I've used. The great thing about using this module with Terraform is being able to run your tests

[elijah]

Ed, this is great tutorial content! Maybe good for a blog post to make it easier for people to find?

--elijah

On Wed, Aug 12, 2020 at 9:00 AM Edward Bartholomew notifications@github.com wrote:

Greetings @only4pri https://github.com/only4pri and thank you for your interest in kitchen-terraform! Bootstrapping a server with Chef is something that can happen naturally with Terraform (without need for kitchen-terraform) as I'll describe below. The great thing about using kitchen-terraform though is that you can easily run Inspec tests to verify everything together - that changes made in either Terraform or your cookbook, didn't break your overall module deployment.

My typical workflow would have the Terraform module bundle the cookbook with a null_resource . In this example, I've named it mycookbook and it exists in a different repo than the Terraform module) . If this cookbook uses other cookbooks, then you'll need to use berks package in this below step rather than just zip it up. Use AWS CLI to upload mycookbook cookbook:

resource "null_resource" "upload_cookbook" { depends_on = ["aws_s3_bucket.mycookbook_bootstrapping"] provisioner "local-exec" { command = <<EOF cd /tmp; rm -rf mycookbook mycookbook.zip; git clone git@github.com:newcontext/mycookbook.git; zip -X -r mycookbook.zip mycookbook -x *.git* aws s3 cp mycookbook.zip s3://${aws_s3_bucket.mycookbook_bootstrapping.id} EOF } }

Create a bootstrapping script to run when server starts:

!/usr/bin/env bash

install prereqs

apt-get update apt-get install -y awscli unzip

install chefdk (use a more current version)

wget https://packages.chef.io/files/current/chefdk/3.0.29/ubuntu/16.04/chefdk_3.0.29-1_amd64.deb dpkg -i chefdk_3.0.29-1_amd64.deb

from /tmp, download wrapper cookbook and unzip it

cd /tmp aws s3 cp --region ${aws_region} s3://${s3_bucket_name}/mycookbook.zip . unzip mycookbook.zip

need to have HOME set to avoid something like: https://github.com/ConnorAtherton/rb-readline/issues/8

export HOME=/tmp

download dep cookbooks and run chef-zero to configure

berks vendor --berksfile mycookbook/Berksfile /tmp/cookbooks echo '${chef_attributes}' > attributes.json chef-client -j attributes.json -z --override-runlist recipe['mycookbook']

Create template for above script (create another one if you want attributes like in this example):

data "template_file" "script" { template = "${file("${path.module}/templates/userdata.sh.tpl")}"

vars = { aws_region = "${data.aws_region.current.name}" s3_bucket_name = "${aws_s3_bucket.mycookbook.id}" chef_attributes = "${data.template_file.chef_attributes.rendered}" } }

When creating the instance, set this template to run via user_data:

resource "aws_instance" "repo" { depends_on = ["null_resource.upload_keys"]

ami = "${var.image_id}" instance_type = "${var.instance_type}" user_data = "${data.template_cloudinit_config.config.rendered}"

There's multiple ways but that's one pattern I've used. The great thing about using this module with Terraform is being able to run your tests

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/newcontext-oss/kitchen-terraform/issues/413#issuecomment-672889077, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAKREZDEMYI7TPXGJFL2LDSAKOBPANCNFSM4P4UDFFQ .

[edwardbartholomew]

Thanks for the encouragement @elijah , that's a great idea! Any suggestions - like do you think would cloudinit be preferable to userdata?

[elijah]

Which one allows for the longest bootstrap script to be used? I seem to recall that there are limits on how many characters user-data can be... or maybe that was cloud-init. Not sure right now. It might be useful to adopt a cloud-init strategy because there are many other tools that are already using that - and one more pattern isn't going to hurt someone who's working through a tutorial to learn what to do. :D

--e

On Wed, Aug 12, 2020 at 9:57 AM Edward Bartholomew notifications@github.com wrote:

Thanks for the encouragement @elijah , that's a great idea! Any suggestions - like do you think would cloudinit be preferable to userdata?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[only4pri]

Thanks for your reply. I Created New "null_resource" to run a simple command on remote machine.

resource "null_resource" "example1" { provisioner "remote-exec" { connection { host = "${optumvm_rhel7fsvm.rhel7ServerA.servername}" bastion_host = "apsr*****" bastion_host_key = "pathofthefil" bastion_port = "22" bastion_user = "root" bastion_private_key = "pathofthefil" private_key = "pathofthefile" } inline = [ "mkdir -p /tmp/iamonetest", ] } }

Can Anyone Explain? what private key is needed here ? and from where to fetch it ?

[KevinBuchs]

Hi @only4pri - I think, but am not sure, that you need to supply your Chef private key.