aws_instance not taking input for metadata_options

[jordan-severance]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.0.8

provider registry.terraform.io/hashicorp/aws v3.61.0

Affected Resource(s)

• aws_instance

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_instance" "test" {
  ami = "ami-0123456abc"
  instance_type = "t2.micro"

  metadata_options {
    http_tokens = "required"
    http_endpoints = "enabled"
  }
}

Debug Output

Panic Output

Expected Behavior

AWS EC2 instance should be created with specified metadata options (http_tokens= required, http_endpoints=enabled)

Actual Behavior

AWS EC2 instance is created with default metadata options (http_tokens= optional, http_endpoints=enabled)

Steps to Reproduce

1. terraform apply

Important Factoids

References

[justinretzolk]

Potentially related: https://github.com/hashicorp/terraform-provider-aws/issues/16781

Hey @jordan-severance 👋 I've seen a couple of similar issues, and am keeping the conversation on the one linked above. I'm not positive that this is the exact same bug, but it presents in a way that makes me wonder if it may be. Someone posted a workaround they found here; can you check to see if that workaround works for you as well?

[jordan-severance]

Hi @justinretzolk, thanks for the reply. It does look similar, but what I'm seeing is distinct from that issue. In the issue you linked, they were having a problem with the default option not being automatically taken, so in their workaround, they specified the value in the metadata options.

My issue is that the default option for http_tokens (default is "optional") is overriding what I specify in my TF script. So I tell it I want http_tokens to be "required", but when the instance is created http_tokens is set as "optional".

[jordan-severance]

I also noticed that I can change the http_tokens value to "required" using AWS CLI after creation. Then I tried to re-run terraform apply to see if the http_tokens value would get changed back to "optional". Terraform did not attempt to change the http_tokens value back.

[justinretzolk]

Hey @jordan-severance, thanks for those updates, that's great information. I've marked this as a bug so that the we can take look into this as soon as time allows.

[MysticalMount]

Hi @jordan-severance Im quite new to this and hunting for issues that need code updates, so if I'm completely wrong please forgive me!

TF Version: 1.0.8 Provider version: 3.63.0

Test instance code:

resource "aws_instance" "test" {
  ami = "ami-02f5781cba46a5e8a"
  instance_type = "t2.micro"

  key_name               = "aya369"
  monitoring             = false
  subnet_id              = data.terraform_remote_state.vpc.outputs.public_subnets[0]

  metadata_options {
    http_tokens = "required"
    http_endpoint = "enabled"
  }
}

Ussing the http_endpoint MetaData option, instance was successfully created with the Metadata options set (http_tokens required). The code does explicitly expect http_endpoint and not http_endpoints but should give you a validation error if you have a typo as your bug report seems to have.

The validation error would appear something similar to:

│ Error: Unsupported argument
│ 
│   on main.tf line 24, in resource "aws_instance" "test":
│   24:     http_endpoints = "enabled"
│ 
│ An argument named "http_endpoints" is not expected here. Did you mean "http_endpoint"?

After correcting this the instance was created with the metadata options as specified (aws ec2 describe-instances):

                    "MetadataOptions": {
                        "State": "applied",
                        "HttpTokens": "required",
                        "HttpPutResponseHopLimit": 1,
                        "HttpEndpoint": "enabled",
                        "HttpProtocolIpv6": "disabled"
                    },

Pinning the version of the provider to version in your bug report (instead of the latest):

AWS Provider version: v3.61.0

                    "MetadataOptions": {
                        "State": "applied",
                        "HttpTokens": "required",
                        "HttpPutResponseHopLimit": 1,
                        "HttpEndpoint": "enabled",
                        "HttpProtocolIpv6": "disabled"
                    },

In both versions the behavior appears to be the same so perhaps you are somehow missing/skipping the validation or indeed it is a typo in your report and therefore I could be missing something?

[jordan-severance]

Hi @MysticalMount, sorry but it appears I had a typo in the report. I do have http_endpoint in my code, not http_endpoints. I apologize.

I will also mention that I am using a module to create the instance. The resource block shown above is in the module, along with some other configs. Then from my main.tf I'm calling the module. I don't know if that would change anything up.

[jordan-severance]

Hello, is there any update on this problem?

[MysticalMount]

From my testing above it would seem there is no issue in the code, perhaps there is some difference that is not reported. Have you retried with the latest TF version or is switching not an option for you?

Shouldnt make any difference in the module as long as its not variabilised [in the module] and you are using the same provider.

[bryan292]

Having the same issue, using terraform is not updatin the previous clusters neither the new ones.

[levijskal22]

10/27/2022 facing the same issue where metadata_options is not added

[haytham0123]

Facing the same issue

[eugeneotto]

I got through this by setting MetadataOptions manually via the AWS CLI, thanks to @jordan-severance's note above. For anyone else who finds this issue, here's roughly the command I used:

aws ec2 create-launch-template-version \
  --launch-template-id lt-01234567890 \
  --source-version 123 \
  --version-description "Your description here" \
  --launch-template-data '{
    "MetadataOptions": {
        "HttpTokens": "required", "HttpProtocolIpv6": "disabled", "InstanceMetadataTags": "disabled"
    }
}'

[justinretzolk]

Hi all 👋 I took another look through the provider code, and wasn't able to find any obvious bug that would cause this. Can someone who is still experiencing this provide debug logs (redacted as needed) as well as a sample Terraform configuration that can be used to reproduce this?

[chrisdenton-ct]

I think I've just run into this issue. The AWS console is now highlighting that IMDSv2 should be set to "required" rather than "optional". However when I try to configure that in Terraform with:

  metadata_options {
    http_tokens = "required"
  }

It seems to have no effect. The manual/AWS CLI workaround doesn't really work for me as I'm using spot instances.

[deverm121986]

Hi @chrisdenton-ct

Below worked for me metadata_options { http_endpoint = "enabled" http_tokens = "required" }

[chrisdenton-ct]

Thanks @deverm121986 - that's functionally the same as my config, isn't it? As "http_endpoint" defaults to "enabled".

[deverm121986]

@chrisdenton-ct Yeah its default enabled, however can you try to set as above and test

[chrisdenton-ct]

Thanks again @deverm121986 - specifying the http_endpoint explicitly doesn't make any difference.

The Terraform output claims it is going to make the desired http_tokens update:

      ~ metadata_options {
          ~ http_endpoint               = "disabled" -> "enabled"
          ~ http_tokens                 = "optional" -> "required"
            # (1 unchanged attribute hidden)
        }

However, I can see using the "aws ec2 describe instances" command that it does no such thing:

                    "MetadataOptions": {
                        "State": "applied",
                        "HttpTokens": "optional",
                        "HttpPutResponseHopLimit": 1,
                        "HttpEndpoint": "enabled",
                        "HttpProtocolIpv6": "disabled",
                        "InstanceMetadataTags": "disabled"
                    },

I'm using version 5.3.0 of the AWS provider, though I can't see anything in the release notes that would indicate a fix has been applied to a more recent version.

[chrisdenton-ct]

Just as an update, in case anyone facing similar issues finds this thread, the cause of my problem was that I was using the resource "aws_spot_instance_request" which now has an advisory in the documentation:

**NOTE AWS strongly discourages the use of the legacy APIs called by this resource. We recommend using the EC2 Instance resource with instance_market_options instead.

When I switched to "aws_instance" with "instance_market_options", the exact same "metadata_options" configuration actually took effect.