Add option to define aws availability zone

[JeanMertz]

I'm having an issue where my provisioner needs to mount an Amazon EBS to my server. Most of the time, this works, but sometimes it errors out, complaining that the EBS is created in a different availability zone and can't be mounted.

I searched the Packer docs/source code, but it doesn't seem to be possible to define an exact availability zone (as opposed to just the region).

The only reference I found was this:

// Region returns the aws.Region object for access to AWS services, requesting
// the region from the instance metadata if possible.
func (c *AccessConfig) Region() (aws.Region, error) {
    if c.RawRegion != "" {
        return aws.Regions[c.RawRegion], nil
    }

    md, err := aws.GetMetaData("placement/availability-zone")
    if err != nil {
        return aws.Region{}, err
    }

    region := strings.TrimRightFunc(string(md), unicode.IsLetter)
    return aws.Regions[region], nil
}

Is this at all possible, or is it possible to add this to Packer? (I can start looking into this, but maybe you know of a quick and easy way to add it)

update it seems that Amazon has decided to create all my nodes in the eu-west-1b today, whereas it added them to eu-west-1a yesterday. So it seems like this can be a big issue with the random assignments.

[mwhooker]

Which provisioner are you using? Provisioners are supposed to be user defined, so I think a user variable would be a better place to define the region.

Do you have an example of your packer template you can share?

[JeanMertz]

@mwhooker Actually, I believe the availability-zone would be defined in the amazon-ebs builder (or any other amazon builder) I would assume?

Take this scenario:

I have a chef-solo provisioner (or a shell script for chef-server) that mounts an existing data EBS. This EBS was initially created in availability-zone eu-west-1a (which is within region eu-west-1. When Packer runs, it needs this mount to be able to provision the AMI correctly.

In my Packer template, I defined the region as eu-west-1. However, since I am unable to set the availability zone, Amazon sometimes starts up the packer-created node in a different zone as the EBS (1b for the node vs 1a for the EBS).

In order to fix this, I need a pre-defined availability-zone in which Packer builds the AMI's.

Does this make any sense?

[JeanMertz]

Also, here's an example packer template for this node:

{
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": ""
  },

  "builders": [
    {
      "type": "amazon-ebs",
      "name": "my-server-aws",
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "region": "eu-west-1",
      "source_ami": "...",
      "instance_type": "m1.medium",
      "ssh_username": "ubuntu",
      "ami_name": "my-server {{isotime | clean_ami_name}}",
      "security_group_id": "..."
    }
  ],

  "provisioners": [
    {
      "type": "file",
      "source": "/tmp/encrypted_data_bag_secret",
      "destination": "/tmp/secret"
    },
    {
      "type": "shell",
      "inline": [
        "sudo mkdir -p /etc/chef",
        "sudo mv /tmp/secret /etc/chef/encrypted_data_bag_secret"
      ]
    },
    {
      "type": "chef-solo",
      "cookbook_paths": ["vendor/cookbooks"],
      "data_bags_path": "data_bags",
      "run_list": ["recipe[my-app-recipe]"],
    }
  ]
}

My proposal would be to add an availability_zone option to the builders section.

[mwhooker]

oh okay, I think I get it.

So that I understand: a part of your provision process attaches an EBS volume. In order to do that you need to make sure the ebs--backed instance packer launches is in the same AZ as your volume?

I will mark this as an enhancement, since it's essentially a new feature, unless anyone disagrees.

I apologize, I don't have too much experience working with EBS, but in the mean time, would a viable alternative be to use a snapshot and create a volume with it as part of the provisioning step? that way you'll always be able to access it and provides better availability guarantees if a region goes down.

I suppose the downside is needing to clean up the volume. Perhaps the amazon builders should accept a list of snapshots to attach as volumes. That way packer can deal with cleaning them up. Thoughts, @mitchellh?

[JeanMertz]

@mwhooker You are correct with your analysis.

Regarding the snapshot, I guess this would work, but the reason we mount the "live" ebs instead of a snapshot, is that we want to make sure we're mounting the latest data, instead of snapshot that could be several hours/days old.

To remedy this, I've changed the provisioning step (in Chef, not in the Packer template) to create a new ebs if the old one can't be found. This is "not an issue", because the EBS isn't really used during provisioning, only when the new AMI is launched will the EBS be used, and at that point, the EBS can be selected manually in the Amazon interface (or with knife).

But it does clutter up the provisioning scripts, and it also a bit less safe, if someone boots up the AMI without the ebs attached, and in a different region than the "data production" ebs, it will automatically create a new - blank - ebs.

[mwhooker]

@JeanMertz if you're able to test #550, I'll be able to merge this in

[mitchellh]

Fixed by #550. Also, I added it to instance runs.

[haf]

Is this in 0.4.0?