Closed patricklucas closed 9 years ago
phinze
commented
9 years ago Hey @patricklucas - sorry this is causing you trouble.
Over in https://github.com/hashicorp/terraform/issues/859 we determined that terraform should ignore root devices, which is the current behavior in master. It looks like your config is declaring a root device - do you get coherent behavior when you only specify the non-root device in your config and let AWS auto-create the other one?
patricklucas
commented
9 years ago If I remove the /dev/sda1 block_device, it does indeed start to work.
But, I need to specify the root device because I need to override its volume type and size—I want gp2/16GB instead of the default standard/8GB.
Is this not possible with Terraform?
phinze
commented
9 years ago The solution we landed on in #859 may have been insufficient. Let me think about this a bit.
threetee
commented
9 years ago I see this behavior even when I don't specify the root device. Here's my config:
resource "aws_instance" "my_instance" {
ami = "${var.my_ami}"
instance_type = "t2.small"
availability_zone = "us-west-2a"
key_name = "my_key"
subnet_id = "${aws_subnet.my_subnet.id}"
block_device = {
device_name = "/dev/sdf"
volume_type = "standard"
volume_size = "100"
delete_on_termination = false
}
}terraform apply forces a new resource, with this relevant output:
block_device.#: "2" => "1"
block_device.2577914260.delete_on_termination: "false" => "0" (forces new resource)
block_device.2577914260.device_name: "/dev/sdf" => "/dev/sdf" (forces new resource)
block_device.2577914260.encrypted: "false" => "<computed>"
block_device.2577914260.snapshot_id: "" => "<computed>"
block_device.2577914260.virtual_name: "" => ""
block_device.2577914260.volume_size: "100" => "100" (forces new resource)
block_device.2577914260.volume_type: "standard" => "standard" (forces new resource)
MerlinDMC
commented
9 years ago In my terraform plan output I got the following block_device mapping shown:
block_device.#: "1" => "1"
block_device.3403969019.delete_on_termination: "" => "1" (forces new resource)
block_device.3403969019.device_name: "" => "xvdb" (forces new resource)
block_device.3403969019.encrypted: "" => "<computed>" (forces new resource)
block_device.3403969019.snapshot_id: "" => "<computed>" (forces new resource)
block_device.3403969019.virtual_name: "" => "ephemeral0" (forces new resource)
block_device.3403969019.volume_size: "" => "<computed>" (forces new resource)
block_device.3403969019.volume_type: "" => "<computed>" (forces new resource)while the statefile from the last apply has the following mapping registered:
"block_device.#": "1",
"block_device.2849382189.delete_on_termination": "false",
"block_device.2849382189.device_name": "",
"block_device.2849382189.encrypted": "false",
"block_device.2849382189.snapshot_id": "snap-01fe7529",
"block_device.2849382189.virtual_name": "",
"block_device.2849382189.volume_size": "16",
"block_device.2849382189.volume_type": "standard",The fact that those numeric ids don't match lets me assume that terraform does think it has to recreate the whole device every time as it is a unknown one.
Can someone point me to the commit where the diff introduced the numeric id for lists?
phinze
commented
9 years ago Thanks for the feedback folks. We'll get this all untangled.
@threetee is your case one where you are adding a block device to an existing instance, or was that /dev/sdf device there on creation?
I think one important issue here for updates is that we're currently marking every attribute of block_devices as ForceNew - https://github.com/hashicorp/terraform/blob/680fa3c0d8022caa8e4fa37071a93f39dbc521e3/builtin/providers/aws/resource_aws_instance.go#L142-L187
This is because the Update function is pretty anemic right now - it only supports passing SourceDestCheck to the AWS ModifyInstance API - https://github.com/hashicorp/terraform/blob/680fa3c0d8022caa8e4fa37071a93f39dbc521e3/builtin/providers/aws/resource_aws_instance.go#L419
@MerlinDMC Same question for you - is this for a new instance or for updates? FWIW we're tracking ephemeral storage support over in #858, though all of this stuff is very intertwined.
So teasing out the different issues we have so far:
MerlinDMC
commented
9 years ago @phinze I created a new instance in a clean environment and then ran terraform plan again
transscript is: https://gist.github.com/MerlinDMC/5bfd28ccd98eded278d9
using terraform 0.3.6 installed via homebrew
config used:
provider "aws" {
region = "eu-west-1"
}
resource "aws_instance" "test" {
ami = "ami-00b11177"
instance_type = "m3.medium"
associate_public_ip_address = true
block_device {
device_name = "xvdb"
virtual_name = "ephemeral0"
delete_on_termination = 1
}
count = 1
key_name = "${var.key_name}"
tags {
Name = "Test Kill Me!"
}
connection {
user = "ubuntu"
key_file = "${var.key_path}"
}
provisioner "remote-exec" {
inline = [
"uname -a",
]
}
}
jschneiderhan
commented
9 years ago I hope I'm not way off here, but while trying to debug this (with a very limited knowledge of Go) it looks like the delete_on_termination value at https://github.com/hashicorp/terraform/blob/master/builtin/providers/aws/resource_aws_instance.go#L513 is being set to false even when the value of the actual resource in AWS is true. If I hard-code true here the hash in the statefile and the hash in the new resource match and a re-creation doesn't occur.
phinze
commented
9 years ago Hey @jschneiderhan - thanks the debugging assist!
I think you're right that there's a problem with delete_on_termination down there. This issue is next on my list of terraform bugs to pick up - I'll add this to my notes and report back here when I make progress. :+1:
jschneiderhan
commented
9 years ago As a workaround, if you specify delete_on_termination = true in the aws_instance instead of relying on the default, it works just fine.
patricklucas
commented
9 years ago @jschneiderhan: That doesn't work for me. Even when explicitly setting it to true, terraform tries to refresh the resources on every run.
jschneiderhan
commented
9 years ago @patricklucas noted - perhaps it's dependent on the version I'm using, 0.3.5, which is a bit older. The following leads to re-creation of the instance:
provider "aws" {
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
region = "${var.region}"
}
resource "aws_instance" "nat" {
ami = "[REDACTED]"
key_name = "[REDACTED]"
instance_type = "m3.medium"
availability_zone = "us-east-1b"
security_groups =["default"]
block_device {
device_name = "/dev/sda1"
volume_size = "150"
#delete_on_termination = true
}
}But this does not:
provider "aws" {
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
region = "${var.region}"
}
resource "aws_instance" "nat" {
ami = "[REDACTED]"
key_name = "[REDACTED]"
instance_type = "m3.medium"
availability_zone = "us-east-1b"
security_groups =["default"]
block_device {
device_name = "/dev/sda1"
volume_size = "150"
delete_on_termination = true
}
}I just installed v0.3.6 and see the same behavior with the above template. It must be conditional on something, but I don't know what.
I need to move on to something else, but if there is anything more I can try to help please let me know.
threetee
commented
9 years ago is your case one where you are adding a block device to an existing instance, or was that /dev/sdf device there on creation?
@phinze sorry, upon re-reading my post, I can see how it wasn't clear exactly how I arrived at the problem. The /dev/sdf device was in fact there on creation. I just ran through the whole process again to ensure I could still replicate, and here are the exact steps to duplicate for me:
-/+ aws_instance.myinstance
ami: "ami-d3adb33f" => "ami-d3adb33f"
availability_zone: "us-west-2a" => "us-west-2a"
block_device.#: "2" => "1"
block_device.2577914260.delete_on_termination: "false" => "0" (forces new resource)
block_device.2577914260.device_name: "/dev/sdf" => "/dev/sdf" (forces new resource)
block_device.2577914260.encrypted: "false" => "<computed>"
block_device.2577914260.snapshot_id: "" => "<computed>"
block_device.2577914260.virtual_name: "" => ""
block_device.2577914260.volume_size: "100" => "100" (forces new resource)
block_device.2577914260.volume_type: "standard" => "standard" (forces new resource)
instance_type: "t2.small" => "t2.small"
key_name: "mykey" => "mykey"
private_dns: "ip-1-2-3-4.us-west-2.compute.internal" => "<computed>"
private_ip: "1.2.3.4" => "<computed>"
public_dns: "" => "<computed>"
public_ip: "" => "<computed>"
security_groups.#: "4" => "4"
security_groups.1887157646: "sg-12345678" => "sg-12345678"
security_groups.2806254740: "sg-87654321" => "sg-87654321"
security_groups.3046043015: "sg-d3adb33f" => "sg-d3adb33f"
security_groups.3548738533: "sg-10101010" => "sg-10101010"
subnet_id: "subnet-efefefef" => "subnet-efefefef"
tags.#: "1" => "1"
tags.Name: "myinstance" => "myinstance"
tenancy: "default" => "<computed>"
user_data: "576ca5e7da108364c385e21f8b230ffd34b33fcf" => "576ca5e7da108364c385e21f8b230ffd34b33fcf"aws_instance.myinstance: Destruction complete
aws_instance.myinstance: Creating...
<snip>I'm running terraform v0.3.6, installed via homebrew on OS X Yosemite.
axelpavageau
commented
9 years ago I'm having the same issue :
-/+ aws_instance.aws-admin
ami: "ami-2278cc55" => "ami-2278cc55"
associate_public_ip_address: "true" => "1"
availability_zone: "eu-west-1b" => "eu-west-1b"
block_device.#: "2" => "1"
block_device.3293284504.delete_on_termination: "false" => "0" (forces new resource)
block_device.3293284504.device_name: "/dev/sdb" => "/dev/sdb" (forces new resource)
block_device.3293284504.encrypted: "false" => "<computed>"
block_device.3293284504.snapshot_id: "" => "<computed>"
block_device.3293284504.virtual_name: "" => ""
block_device.3293284504.volume_size: "50" => "50" (forces new resource)
block_device.3293284504.volume_type: "gp2" => "gp2" (forces new resource)
ebs_optimized: "false" => "0"
instance_type: "t1.micro" => "t1.micro"
[...] My current workaround is to create the instance with the root block device defined, then remove it for subsequent runs.
If you always want the same root device spec, you can also make it your account default. When you launch an instance with Terraform without defining your root device it will use those settings.
pmoust
commented
9 years ago @patricklucas this is not an option however as it defies the IaC purpose.
phinze
commented
9 years ago Hey folks, sorry for all the trouble with this.
We're seeing this pattern of sadness in several different places where the backing API provides a collection of objects in which a subset has certain special properties - particularly where a subset of the collection can be auto-populated while the rest is managed.
In this case, the collection of block_devices can be auto-populated with a root device, or the root device can be managed by config, all while the remainder of block devices can be optionally specified.
Today I'm working on a candidate solution for this that I'd love to get feedback on in case it's not the right direction.
Basically I'd like to try splitting out the root device into a separate sub-resource like this:
resource "aws_instance" "foo" {
ami = "ami-55a7ea65"
instance_type = "m1.small"
root_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
block_device {
device_name = "/dev/sdb"
volume_size = 11
}
block_device {
device_name = "/dev/sdc"
volume_size = 12
}
// ...
}I believe this will make the behavior much clearer. Let me know what you think!
joegoggins
commented
9 years ago +1 on a root_block_device sub-resource. This would result in a large amount of bliss in my world. :)
Would it be safe to default the device_name to "/dev/sda1" instead of require that a user specify it? The EC2/EBS documentation makes it pretty clear that this is always the default for linux boxes and would prevent mistakes (like the one I just made) where a user specifies /dev/sda instead of /dev/sda1. Also, for this root_block_device sub-resource--I assume it would force the volume_type to "gp2", yea?
In the meantime, until this ships, I'm going with this approach to making the root volume larger:
block_device {
device_name = "/dev/sda1"
volume_size = 100
volume_type = "gp2"
delete_on_termination = true
}Please let me know if there is a better way for now (with Terraform 0.3.5 or 0.3.6). Thanks!
pmoust
commented
9 years ago @phinze suggested approach is very fitting. +1.
patricklucas
commented
9 years ago @phinze: sgtm
I concur with @joegoggins that device_name can likely be safely defaulted to /dev/sda1, though I believe that volume_size and volume_type should fall back to your account defaults as is the current behavior.
ghost
commented
4 years ago I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.
I am unable to use latest (c18b01fa) Terraform to manage AWS instances because it tries to destroy and re-create them on every invocation. I think it has something to with my block device specifications. I have already launched quite a few instances with Terraform, but I can't make any modifications lest they get destroyed.
Here is the Terraform config and output: