Plugin still tries to create keypair even when using Session Manager

[firke]

Overview of the Issue

I've tried to build an image only using SSM Session Manager, CICD system does not have IAM permissions to create keypairs in AWS, so even though the project is only using SSM it tries to create keypairs for SSH.

Reproduction Steps

Have the following lines in config:

  ssh_interface               = "session_manager"
  communicator                = "ssh"

Run packer build.

Plugin and Packer version

Packer v1.8.6

Simplified Packer Buildfile

source "amazon-ebs" "ami" {
  skip_create_ami             = true
  region                      = var.aws_region
  subnet_id                   = var.aws_subnet_map[var.aws_region]
  instance_type               = var.aws_ec2_instance_type_map[var.os_type]
  source_ami                  = var.ami
  ami_name                    = "packer_aws_example"
  iam_instance_profile        = "AmazonSSMManagedInstanceCore"
  ssh_username                = "core"
  ssh_interface               = "session_manager"
  communicator                = "ssh"
}

build {
  name = "flatcar"
  sources = [
    "source.amazon-ebs.ami"
  ]
}

Log Fragments and crash.log files

$ packer build -var-file flatcar.pkrvars.hcl .
flatcar.amazon-ebs.ami: output will be in this color.
==> flatcar.amazon-ebs.ami: Prevalidating any provided VPC information
==> flatcar.amazon-ebs.ami: Prevalidating AMI Name: packer_aws_example_1679919306
    flatcar.amazon-ebs.ami: Found Image ID: ami-0ba1aac16966efa09
==> flatcar.amazon-ebs.ami: Creating temporary keypair: packer_642188ca-c3d8-0f86-417b-17c7eb878f57
Build 'flatcar.amazon-ebs.ami' errored after 2 minutes 25 seconds: Error creating temporary keypair: retry count exhausted. Last err: UnauthorizedOperation: You are not authorized to perform this operation.

[lbajolet-hashicorp]

Hi @firke,

Even when interacting with your instance through SSM, we still use SSH for sending commands/files over to the instance for provisioning. In the plugin, SSM is used for providing a tunnel for connecting to the instance, not directly to open a ssh connection. This is preferred in order not to have to expose the instance to the web with a public IP and the necessary VPC/SG/Route configuration. By default, if nothing is provided for authentication (SSH private key, password), we create a new keypair, and use the AWS API to do this, hence the error you're experiencing.

So for this problem, as it stands, you have two solutions:

1. Add a ssh_password to your config, in this case, ssh will attempt to connect by providing a password
2. Add a ssh_private_key_file to your config, and then ssh will connect using this key.

Note that both aren't mutually exclusive, and we will attempt to use both in sequence to authenticate.

~Looking at the code for the plugin however, I'm not sure it won't create a key if a password is specified, which may be an omission on our part.~ EDIT: the feature is indeed implemented, if a password is specified, no temporary key is specified.

I'll look into this, but in the meantime, if you do specify a private key, the AWS temporary key won't be generated, and you should not encounter the issue anymore, can you confirm this is the case?

Also for reference, if you are using SSM and a private key, the public key will be sent over to the SSM agent (using this API call: https://docs.aws.amazon.com/ec2-instance-connect/latest/APIReference/API_SendSSHPublicKey.html), and will register it for authentication when ssh attempts to connect, you should not need to add that key to your instance manually.

EDIT: regarding password authentication, it is disabled by default on Amazon base AMIs, so this should not be the preferred auth method here.

[firke]

Hey @lbajolet-hashicorp, thank you for clarifying it! I did notice in Cloudtrail that the user opened up two ports - 22 and a random one, realised a port forwarding is used there.

As a note, I did encounter an issue with supplying my own key - the API call SendSSHPublicKey was always successful, but it never did anything since I've used a flatcar OS, which isn't bundled with Instance Connect, seems the only ones so far with it are Ubuntu/Amazon Linux 2. So in my case it never sent the public key through and then SSH failed to connect. I will see if it's fairly straight-forward to include it at the start-up. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-connect-set-up.html#ec2-instance-connect-install

Not sure if there is any easy fix for the instance connect bug and the original issue is resolved, happy for this to be closed if it's good with you.

[lbajolet-hashicorp]

Oh true, this may indeed be a problem if you don't have the AWS daemons installed on that OS, sshd will never know about your public key, and connection attempts will get rejected.

I presume in this case you can maybe add the public key that matches the private one you want to use to the instance manually, but this is outside the scope of this plugin I'm afraid, since it is built on the premise that you are able to talk to your instance through SSH/WinRM (if using Windows).

If my message helps you find a solution, we can close it. I'll do that, and if you have an update for it later, feel free to reopen it, or open a new one, as you prefer.

Good luck with your setup!