search

hashicorp / packer-plugin-googlecompute

Packer plugin for Google Compute Builder
https://www.packer.io/docs/builders/googlecompute
Mozilla Public License 2.0
23 stars 51 forks source link

Using IAP for SSH access does not work #195

Open janneronkko opened 8 months ago

janneronkko commented 8 months ago

Overview of the Issue

SSH through IAP uses incorrect IP for connecting.

The tunnel exists and is bound to localhost and can be used with ssh -p <port> localhost (you can also replace localhost with 127.0.0.1 or ::1 so both IPv4 and IPv6 works)

Reproduction Steps

Set use_iap = true for googlecompute resource:

source "googlecompute" "ubuntu2204" {
  use_iap                 = true
  ...
}

Plugin and Packer version

Packer v1.9.4 googlecompute: 1.1.3

Simplified Packer Buildfile

packer {
  required_plugins {
    googlecompute = {
      source  = "github.com/hashicorp/googlecompute"
      version = ">= 1.1.3"
    }
  }
}

source "googlecompute" "ubuntu" {
  project_id              = "gcp-project-id"
  use_iap                 = true

  source_image_family     = "ubuntu-2204-lts"
  source_image_project_id = [
    "ubuntu-os-cloud",
  ]
  zone                    = "europe-north1-a"

  tags                    = [
    "packer",
  ]

  image_family            = "custom-ubuntu-2204"
  image_name              = "custom-ubuntu-2204-v1"

  machine_type            = "e2-standard-2"
  subnetwork              = "vm-image-build-europe-north1"
  disk_size               = "10"
  disk_type               = "pd-ssd"
  use_os_login            = false
  ssh_username            = "ubuntu"
}

build {
  sources = ["source.googlecompute.ubuntu"]

  provisioner "shell" {
    inline = ["echo test"]
  }
}

Operating system and Environment details

/etc/os-release:

NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo

Log Fragments and crash.log files

2023/10/27 15:23:06 [INFO] Packer version: 1.9.4 [go1.21.0 linux amd64]
2023/10/27 15:23:06 [TRACE] discovering plugins in /usr/bin
2023/10/27 15:23:06 [TRACE] discovering plugins in .
2023/10/27 15:23:06 [TRACE] discovering plugins in /home/janne/.config/packer/plugins
2023/10/27 15:23:06 [INFO] Discovered potential plugin: googlecompute = /home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.2_x5.0_linux_amd64
2023/10/27 15:23:06 [INFO] Discovered potential plugin: googlecompute = /home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64
2023/10/27 15:23:06 [INFO] found external [-packer-default-plugin-name-] builders from googlecompute plugin
2023/10/27 15:23:06 [INFO] found external [export import] post-processors from googlecompute plugin
2023/10/27 15:23:06 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/10/27 15:23:06 [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/janne/.packerconfig
2023/10/27 15:23:06 [WARN] Config file doesn't exist: /home/janne/.packerconfig
2023/10/27 15:23:06 [INFO] Setting cache directory: /home/janne/.cache/packer
2023/10/27 15:23:06 [TRACE] listing potential installations for "github.com/hashicorp/googlecompute" that match ">= 1.1.3". plugingetter.ListInstallationsOptions{FromFolders:[]string{"/usr/bin", ".", "/home/janne/.config/packer/plugins"}, BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"linux", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc0009a2500)}}}}
2023/10/27 15:23:06 [TRACE] version "v1.1.2" of file "/home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.2_x5.0_linux_amd64" does not match constraint ">= 1.1.3"
2023/10/27 15:23:06 [TRACE] Found the following "github.com/hashicorp/googlecompute" installations: [{/home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 v1.1.3}]
2023/10/27 15:23:06 [INFO] found external [-packer-default-plugin-name-] builders from googlecompute plugin
2023/10/27 15:23:06 [INFO] found external [export import] post-processors from googlecompute plugin
2023/10/27 15:23:06 [INFO] Starting external plugin /home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 start builder -packer-default-plugin-name-
2023/10/27 15:23:06 Starting plugin: /home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 []string{"/home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64", "start", "builder", "-packer-default-plugin-name-"}
2023/10/27 15:23:06 Waiting for RPC address for: /home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64
2023/10/27 15:23:06 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:06 Plugin address: unix /tmp/packer-plugin645941989
2023/10/27 15:23:06 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:06 Waiting for connection...
2023/10/27 15:23:06 Received unix RPC address for /home/janne/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64: addr is /tmp/packer-plugin645941989
2023/10/27 15:23:06 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:06 Serving a plugin connection...
2023/10/27 15:23:06 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:06 [TRACE] starting builder -packer-default-plugin-name-
2023/10/27 15:23:06 [INFO] Starting internal plugin packer-provisioner-shell
2023/10/27 15:23:06 Starting plugin: /usr/bin/packer []string{"/usr/bin/packer", "plugin", "packer-provisioner-shell"}
2023/10/27 15:23:06 Waiting for RPC address for: /usr/bin/packer
2023/10/27 15:23:06 packer-provisioner-shell plugin: [INFO] Packer version: 1.9.4 [go1.21.0 linux amd64]
2023/10/27 15:23:06 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/10/27 15:23:06 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/janne/.packerconfig
2023/10/27 15:23:06 packer-provisioner-shell plugin: [WARN] Config file doesn't exist: /home/janne/.packerconfig
2023/10/27 15:23:06 packer-provisioner-shell plugin: [INFO] Setting cache directory: /home/janne/.cache/packer
2023/10/27 15:23:06 packer-provisioner-shell plugin: args: []string{"packer-provisioner-shell"}
2023/10/27 15:23:06 packer-provisioner-shell plugin: Plugin address: unix /tmp/packer-plugin1363397994
2023/10/27 15:23:06 packer-provisioner-shell plugin: Waiting for connection...
2023/10/27 15:23:06 Received unix RPC address for /usr/bin/packer: addr is /tmp/packer-plugin1363397994
2023/10/27 15:23:06 packer-provisioner-shell plugin: Serving a plugin connection...
2023/10/27 15:23:06 Build debug mode: false
2023/10/27 15:23:06 Force build: false
2023/10/27 15:23:06 On error: 
2023/10/27 15:23:06 Waiting on builds to complete...
2023/10/27 15:23:06 Starting build run: googlecompute.ubuntu
2023/10/27 15:23:06 Running builder: 
2023/10/27 15:23:06 [INFO] (telemetry) Starting builder googlecompute.ubuntu
googlecompute.ubuntu: output will be in this color.

2023/10/27 15:23:06 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:06 [INFO] Requesting Google token via GCE API Default Client Token Source...
2023/10/27 15:23:06 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:06 [INFO] Instantiating GCE client...
2023/10/27 15:23:06 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:06 [INFO] Instantiating OS Login client...
==> googlecompute.ubuntu: Checking image does not exist...
==> googlecompute.ubuntu: Creating temporary RSA SSH key for instance...
==> googlecompute.ubuntu: no persistent disk to create
==> googlecompute.ubuntu: Using image: ubuntu-2204-jammy-v20231025a
==> googlecompute.ubuntu: Creating instance...
2023/10/27 15:23:11 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:11 [DEBUG] Adding metadata during instance creation...
    googlecompute.ubuntu: Loading zone: europe-north1-a
    googlecompute.ubuntu: Loading machine type: e2-standard-2
2023/10/27 15:23:11 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:11 [DEBUG] using google-managed encryption key for boot disk
    googlecompute.ubuntu: Requesting instance creation...
    googlecompute.ubuntu: Waiting for creation operation to complete...
2023/10/27 15:23:15 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:15 Retryable error: retrying for state DONE, got RUNNING
2023/10/27 15:23:17 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:17 Retryable error: retrying for state DONE, got RUNNING
2023/10/27 15:23:19 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:19 Retryable error: retrying for state DONE, got RUNNING
2023/10/27 15:23:21 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:21 Retryable error: retrying for state DONE, got RUNNING
    googlecompute.ubuntu: Instance has been created!
==> googlecompute.ubuntu: Waiting for the instance to become running...
    googlecompute.ubuntu: IP: 34.88.104.24
==> googlecompute.ubuntu: Step Launch IAP Tunnel...
2023/10/27 15:23:24 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:24 Finding an available TCP port for IAP proxy
2023/10/27 15:23:24 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:24 Found available port: 8443 on IP: 0.0.0.0
2023/10/27 15:23:24 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:24 Setting up proxy to listen on localhost at 8443
2023/10/27 15:23:24 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:24 Creating tunnel launch script with args []string{"compute", "start-iap-tunnel", "packer-653babaa-fb1c-5c92-a09a-f176521488fc", "22", "--local-host-port=localhost:8443", "--zone", "europe-north1-a", "--project", "vincit-jenkins"}
2023/10/27 15:23:24 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:24 [INFO] (google): Prepending inline gcloud setup script with #!/bin/sh
2023/10/27 15:23:26 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:26 stderr: WARNING:
2023/10/27 15:23:27 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:27 stderr:
2023/10/27 15:23:28 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:28 stderr: To increase the performance of the tunnel, consider installing NumPy. For instructions,
2023/10/27 15:23:29 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:29 stderr: please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth
2023/10/27 15:23:30 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:30 stderr:
2023/10/27 15:23:31 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:31 stderr: Testing if tunnel connection works.
2023/10/27 15:23:32 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:23:32 stderr: ERROR: (gcloud.compute.start-iap-tunnel) While checking if a connection can be made: Error while connecting [4047: 'Failed to lookup instance'].

The testing if tunnel connection works repeats some time until the tunnel is up:

2023/10/27 15:24:06 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:24:06 stderr: Testing if tunnel connection works.
2023/10/27 15:24:29 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:24:29 No error detected after tunnel launch; continuing...
2023/10/27 15:24:29 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:24:29 Using host value: localhost
==> googlecompute.ubuntu: Using SSH communicator to connect: localhost
2023/10/27 15:24:29 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:24:29 [INFO] Waiting for SSH, up to timeout: 5m0s
2023/10/27 15:24:29 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:24:29 Using host value: localhost
==> googlecompute.ubuntu: Waiting for SSH to become available...
2023/10/27 15:24:32 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:24:32 [DEBUG] TCP connection to SSH ip/port failed: dial tcp 10.1.9.31:8443: connect: no route to host
2023/10/27 15:24:37 packer-plugin-googlecompute_v1.1.3_x5.0_linux_amd64 plugin: 2023/10/27 15:24:37 Using host value: localhost

Packer keeps trying to connect to IP 10.1.9.31 port 8443 for some time until it times out.

My local network is 10.1.9.0/24 but the IP 10.1.9.31 is not used by my computer (or any other device on the network).

$ ss -p -t -l -n | grep 8443
LISTEN                   0                        1                                                                          127.0.0.1:8443                                            0.0.0.0:*                       users:(("python",pid=232484,fd=8))

$ $ ssh -p 8443 localhost
The authenticity of host '[localhost]:8443 ([::1]:8443)' can't be established.
ED25519 key fingerprint is SHA256:w++VHb4NH55FwMEzuOoAv+JdLLn64JqNUHXnBNZRZug.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])?
wyardley commented 5 months ago

I'm seeing this as well. I commented in https://github.com/hashicorp/packer-plugin-googlecompute/issues/37#issuecomment-1921864702

Oddly, the same config was working for me yesterday.