AWS: packer cant find configuration key for instance_metadata_tag option

[mitter91]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

here manifest file i used without changes:

{
  "variables": {
    "env": "{{ env `ENV` }}",
    "f5_type": "{{ env `F5_TYPE` }}",
    "release_version": "{{ env `RELEASE_VERSION` }}",
    "ssh_key_name": "{{ env `SSH_KEYPAIR` }}"
  },
  "builders": [
    {
      "type": "amazon-ebs",
      "region": "eu-central-1",
      "skip_create_ami": true,
      "ssh_pty": true,
      "vpc_filter": {
        "filters": {
          "tag:Name": "obo",
          "isDefault": "false"
        }
      },
      "subnet_filter": {
        "filters": {
              "tag:Name": "obo:lg1a-pod:app-mgmt-001"
        },
        "most_free": "true",
        "random": "false"
      },
      "security_group_filter": {
        "filters": {
          "tag:Name": "cs-f5*"
        }
      },
      "source_ami_filter": {
        "filters": {
          "tag:Name": "F5_BIG-IP_{{ user `ami_type` }}_R4.??_LAB-ECX_OBO"
        },
        "owners": [
          "self"
        ],
        "most_recent": true
      },
      "metadata_options": {
        "http_endpoint": "enabled",
        "http_tokens": "optional",
        "http_put_response_hop_limit": 1,
        "instance_metadata_tags": "enabled"
      },
      "instance_type": "m5.xlarge",
      "encrypt_boot": true,
      "ssh_username": "admin",
      "ssh_timeout": "10m",
      "ami_name": "{{ user `f5_type` }}",
      "ami_description": "F5 BIG-IP AMI",
      "ena_support": true,
      "ssh_keypair_name": "{{ user `ssh_key_name` }}",
      "ssh_agent_auth": true,
      "run_tags": {
        "Name": "F5_BIG-IP_Backup_Packer_Builder",
        "Backup_Packer": "true",
        "Env": "{{ user `env` }}",
        "Service": "{{ user `f5_type` }}",
        "Release": "{{ user `release_version` }}",
        "Terminator_Postpone": "true"
      }
    }
  ],
  "provisioners": [
    {
      "type": "shell-local",
      "pause_before": "120s",
      "command": "ansible-inventory -i inventory/{{ user `env` }}/aws_ec2.yml --graph"
    },
    {
      "type": "ansible",
      "playbook_file": "f5.yml",
      "inventory_file": "inventory/{{ user `env` }}/aws_ec2.yml",
      "extra_arguments": [
        "-t",
        "{{ user `tags` }}"
      ]
    }
  ]
}

amazon plugin updated to 1.0.9 version

logout from packer:

Error: Failed to prepare build: "amazon-ebs"

1 error occurred:

• unknown configuration key: '"metadata_options.instance_metadata_tags"'

[nywilken]

Hi @mitter91 thanks for reaching out. I updated your description to format the provided template. I see the feature you are referring to was introduced in v1.0.9 of the Amazon plugin. Which you have installed. However, the provided error seems to indicate that the plugin being used by Packer might not be the latest.

How did you install the latest Amazon plugin release?

Since you are using JSON Packer will use the plugin versions bundled with Packer and not the plugins installed via packer init or packer plugins. We are currently working on fixing the following two bugs, which may be relevant in this case #11712 #11696 #11697. If these bugs are affecting your use case you can install the latest version of the Amazon plugin by following the manual install steps here.

[nywilken]

I'm going to mark this issue as an Amazon plugin issue, which will automatically transfer it to its respective repo. Please use the information for the automated comment to follow the new issue and continue the conversation there.

[github-actions[bot]]

This issue has been migrated to https://github.com/hashicorp/packer-plugin-amazon/issues/214 due to the Packer Plugin split.

Please follow the new issue for updates.

[mitter91]

Heey, @nywilken, I've tried packer 1.7.10 and 1.8.0 installed from zip file, also for upgrade amazon plugin to the newest version i've used: packer init -upgrade amazon.json.pkr.hcl

  required_plugins {
    amazon = {
      version = ">= 1.0.9"
      source = "github.com/hashicorp/amazon"
    }
  }
}

and received message Installed plugin github.com/hashicorp/amazon v1.0.9 in "/home/jenkins/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.0.9_x5.0_linux_amd64"

[nywilken]

and received message Installed plugin github.com/hashicorp/amazon v1.0.9 in "/home/jenkins/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.0.9_x5.0_linux_amd64"

Thanks for the quick reply @mitter91. You are using Packer init to install the plugin which is great. The issue is that only HCL2 templates will use the installed plugin. JSON templates will continue to use the version of the Amazon plugin that is bundled with Packer, which is pinned to v1.0.6. Since you are already using HCL to install the plugin I recommend migrating your JSON template to HCL by running packer hcl2_upgrade sourcetemplate.json and adding the required plugins block to it.

packer {
  required_plugins {
    amazon = {
      version = ">= 1.0.9"
      source = "github.com/hashicorp/amazon"
    }
  }
}

variable "env" {
  type    = string
  default = "${env("ENV")}"
}

variable "f5_type" {
  type    = string
  default = "${env("F5_TYPE")}"
}

variable "release_version" {
  type    = string
  default = "${env("RELEASE_VERSION")}"
}

variable "ssh_key_name" {
  type    = string
  default = "${env("SSH_KEYPAIR")}"
}

variable "ami_type" {
  type = string
  default = "somevalue"
}

data "amazon-ami" "autogenerated_1" {
  filters = {
    "tag:Name" = "F5_BIG-IP_${var.ami_type}_R4.??_LAB-ECX_OBO"
  }
  most_recent = true
  owners      = ["self"]
  region      = "eu-central-1"
}

source "amazon-ebs" "autogenerated_1" {
  ami_description = "F5 BIG-IP AMI"
  ami_name        = "${var.f5_type}"
  ena_support     = true
  encrypt_boot    = true
  instance_type   = "m5.xlarge"
  metadata_options {
    http_endpoint               = "enabled"
    http_put_response_hop_limit = 1
    http_tokens                 = "optional"
    instance_metadata_tags      = "enabled"
  }
  region = "eu-central-1"
  run_tags = {
    Backup_Packer       = "true"
    Env                 = "${var.env}"
    Name                = "F5_BIG-IP_Backup_Packer_Builder"
    Release             = "${var.release_version}"
    Service             = "${var.f5_type}"
    Terminator_Postpone = "true"
  }
  security_group_filter {
    filters = {
      "tag:Name" = "cs-f5*"
    }
  }
  skip_create_ami  = true
  source_ami       = "${data.amazon-ami.autogenerated_1.id}"
  ssh_agent_auth   = true
  ssh_keypair_name = "${var.ssh_key_name}"
  ssh_pty          = true
  ssh_timeout      = "10m"
  ssh_username     = "admin"
  subnet_filter {
    filters = {
      "tag:Name" = "obo:lg1a-pod:app-mgmt-001"
    }
    most_free = "true"
    random    = "false"
  }
  vpc_filter {
    filters = {
      isDefault  = "false"
      "tag:Name" = "obo"
    }
  }
}

build {
  sources = ["source.amazon-ebs.autogenerated_1"]

  provisioner "shell-local" {
    command      = "ansible-inventory -i inventory/${var.env}/aws_ec2.yml --graph"
    pause_before = "2m0s"
  }

  provisioner "ansible" {
    extra_arguments = ["-t", "${var.tags}"]
    inventory_file  = "inventory/${var.env}/aws_ec2.yml"
    playbook_file   = "f5.yml"
  }

}

[mitter91]

@nywilken ohh, understood, thanks a lot, i'll try that

[mitter91]

@nywilken I've converted manifest now it's: An argument named "instance_metadata_tags" is not expected here.

it should be not in metadata_options block? or even after packer init, i do not use new amazon plugin version, that was installed can i specify it somehow

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.