Packer failed to run target: googlecloud

yaronkaikov commented 1 week ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

When filing a bug, please include the following headings if possible. Any example text in this template can be deleted.

Overview of the Issue

Trying to build a custom googlecompute target after upgrading packager from v1.9.4-1 to v1.11.0-1 and it failed with the following error:

[2024-06-24T08:00:20.033Z] Error: Failed to initialize build "gce"
[2024-06-24T08:00:20.034Z] The builder googlecompute is unknown by Packer, and is likely part of a plugin
[2024-06-24T08:00:20.034Z] that is not installed.
[2024-06-24T08:00:20.034Z] You may find the needed plugin along with installation instructions documented
[2024-06-24T08:00:20.034Z] on the Packer integrations page.
[2024-06-24T08:00:20.034Z] 
[2024-06-24T08:00:20.034Z] https://developer.hashicorp.com/packer/integrations?filter=googlecompute

We are using a docker image to run packer and we have all plugins installed

[yaronkaikov@london]~/git/scylla-pkg (update-packer)$ ./tools/packaging/dpackager 
root@london:/home/yaronkaikov/git/scylla-pkg# packer version
Packer v1.11.0
root@london:/home/yaronkaikov/git/scylla-pkg# packer plugins installed
/root/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.3.2_x5.0_linux_amd64
/root/.config/packer/plugins/github.com/hashicorp/azure/packer-plugin-azure_v2.1.7_x5.0_linux_amd64
/root/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.4_x5.0_linux_amd64

Reproduction Steps

Steps to reproduce this issue

Packer version

Packer v1.11.0

Simplified Packer Template

{
      "name": "gce",
      "type": "googlecompute",
      "source_image_family": "{{user `source_image_family`}}",
      "ssh_username": "{{user `ssh_username`}}",
      "ssh_timeout": "6m",
      "ssh_read_write_timeout": "5m",
      "project_id": "{{user `project_id`}}",
      "zone": "{{user `zone`}}",
      "image_storage_locations": ["{{user `image_storage_location`}}"],
      "machine_type": "{{user `instance_type`}}",
      "metadata": {"block-project-ssh-keys": "TRUE"},
      "image_family": "scylla",
      "image_name": "{{user `image_name`| clean_resource_name}}",
      "image_description": "Official ScyllaDB image v-{{user `scylla_version`| clean_resource_name}}",
      "use_internal_ip": false,
      "preemptible": true,
      "omit_external_ip": false,
      "disk_size": 30,
      "image_labels":  {
          "scylla_version": "{{user `scylla_full_version`| clean_resource_name}}",
          "scylla_machine_image_version": "{{user `scylla_machine_image_version`| clean_resource_name}}",
          "scylla_python3_version": "{{user `scylla_python3_version`| clean_resource_name}}",
          "user_data_format_version": "3",
          "creation_timestamp": "{{user `creation_timestamp`| clean_resource_name}}",
          "branch": "{{user `branch`| clean_resource_name}}",
          "operating_system": "{{user `operating_system`| clean_resource_name}}",
          "scylla_build_sha_id": "{{user `scylla_build_sha_id`| clean_resource_name}}",
          "arch": "{{user `arch`| clean_resource_name}}",
          "build_tag": "{{user `build_tag`| clean_resource_name}}",
          "build_mode": "{{user `build_mode`| clean_resource_name}}"
      },
      "labels": {
        "keep": 1,
        "keep_action": "terminate"
      }
    },

Operating system and Environment details

OS, Architecture, and any other information you can provide about the environment.

Log Fragments and crash.log files

Include appropriate log fragments. If the log is longer than a few dozen lines, please include the URL to the gist of the log or use the Github detailed format instead of posting it directly in the issue.

Set the env var PACKER_LOG=1 for maximum log detail.

nywilken commented 1 week ago

Hi @yaronkaikov thanks for reaching out. I'll pull down the latest Packer Docker full image, which contains all the plugins installed to ensure we can build properly. I'm not sure what is packager, are you using the Packer Docker images we release or is this a custom container?

I would check that the user running Packer has access to the plugin directory /root/.config/packer/plugins/ and that each plugin is executed

To help me better understand what might be happening here could you set the PACKER_LOG=1 environment variable when executing the container to capture the plugin loading output. The logs will provide information on what plugins have been discovered, loaded, and registered with Packer.

~>  docker run -v `pwd`:/workspace -w /workspace hashicorp/packer:full-1.11.0 plugins installed 
/root/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.3.2_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.1_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/azure/packer-plugin-azure_v2.1.4_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.9_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.4_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/qemu/packer-plugin-qemu_v1.1.0_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/vagrant/packer-plugin-vagrant_v1.1.2_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/virtualbox/packer-plugin-virtualbox_v1.0.5_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/vmware/packer-plugin-vmware_v1.0.11_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/vsphere/packer-plugin-vsphere_v1.3.0_x5.0_linux_arm64

~>  docker run -v `pwd`:/workspace -w /workspace -eGOOGLE_PROJECT_ID=$GOOGLE_PROJECT_ID hashicorp/packer:full-1.11.0 validate googlecompute-ubuntu-shell.json
The configuration is valid.

yaronkaikov commented 1 week ago

Hi @yaronkaikov thanks for reaching out. I'll pull down the latest Packer Docker full image, which contains all the plugins installed to ensure we can build properly. I'm not sure what is packager, are you using the Packer Docker images we release or is this a custom container? @nywilken We are using a custom docker image, note that the same container can successfully build both AWS and Azure (under the same configuration file)

I would check that the user running Packer has access to the plugin directory /root/.config/packer/plugins/ and that each plugin is executed

here is the output running from our image (it seems ok)

root@london:~/.config/packer/plugins# ls -larR
.:
total 0
drwxr-xr-x 3 root root 23 Jun 24 09:52 github.com
drwxr-xr-x 3 root root 73 Jun 24 09:52 ..
drwxr-xr-x 3 root root 24 Jun 24 09:52 .

./github.com:
total 0
drwxr-xr-x 5 root root 54 Jun 24 09:53 hashicorp
drwxr-xr-x 3 root root 24 Jun 24 09:52 ..
drwxr-xr-x 3 root root 23 Jun 24 09:52 .

./github.com/hashicorp:
total 0
drwxr-xr-x 2 root root 134 Jun 24 09:52 googlecompute
drwxr-xr-x 2 root root 118 Jun 24 09:53 azure
drwxr-xr-x 2 root root 120 Jun 24 09:53 amazon
drwxr-xr-x 3 root root  23 Jun 24 09:52 ..
drwxr-xr-x 5 root root  54 Jun 24 09:53 .

./github.com/hashicorp/googlecompute:
total 31280
-rw-r--r-- 1 root root       64 Jun 24 09:52 packer-plugin-googlecompute_v1.1.4_x5.0_linux_amd64_SHA256SUM
-rwxr-xr-x 1 root root 32026624 Jun 24 09:52 packer-plugin-googlecompute_v1.1.4_x5.0_linux_amd64
drwxr-xr-x 5 root root       54 Jun 24 09:53 ..
drwxr-xr-x 2 root root      134 Jun 24 09:52 .

./github.com/hashicorp/azure:
total 39160
-rw-r--r-- 1 root root       64 Jun 24 09:53 packer-plugin-azure_v2.1.7_x5.0_linux_amd64_SHA256SUM
-rwxr-xr-x 1 root root 40095744 Jun 24 09:53 packer-plugin-azure_v2.1.7_x5.0_linux_amd64
drwxr-xr-x 5 root root       54 Jun 24 09:53 ..
drwxr-xr-x 2 root root      118 Jun 24 09:53 .

./github.com/hashicorp/amazon:
total 44636
-rw-r--r-- 1 root root       64 Jun 24 09:53 packer-plugin-amazon_v1.3.2_x5.0_linux_amd64_SHA256SUM
-rwxr-xr-x 1 root root 45703168 Jun 24 09:53 packer-plugin-amazon_v1.3.2_x5.0_linux_amd64
drwxr-xr-x 5 root root       54 Jun 24 09:53 ..
drwxr-xr-x 2 root root      120 Jun 24 09:53 .

To help me better understand what might be happening here could you set the PACKER_LOG=1 environment variable when executing the container to capture the plugin loading output. The logs will provide information on what plugins have been discovered, loaded, and registered with Packer.

~>  docker run -v `pwd`:/workspace -w /workspace hashicorp/packer:full-1.11.0 plugins installed 
/root/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.3.2_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.1_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/azure/packer-plugin-azure_v2.1.4_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.9_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/googlecompute/packer-plugin-googlecompute_v1.1.4_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/qemu/packer-plugin-qemu_v1.1.0_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/vagrant/packer-plugin-vagrant_v1.1.2_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/virtualbox/packer-plugin-virtualbox_v1.0.5_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/vmware/packer-plugin-vmware_v1.0.11_x5.0_linux_arm64
/root/.config/packer/plugins/github.com/hashicorp/vsphere/packer-plugin-vsphere_v1.3.0_x5.0_linux_arm64

~>  docker run -v `pwd`:/workspace -w /workspace -eGOOGLE_PROJECT_ID=$GOOGLE_PROJECT_ID hashicorp/packer:full-1.11.0 validate googlecompute-ubuntu-shell.json
The configuration is valid.

Running those commands indeed show some errors (although it also in AWS which we manage to build):

[yaronkaikov@london]~/git/scylla-machine-image/packer (master)$ docker run -v `pwd`:/workspace -w /workspace -eGOOGLE_PROJECT_ID=$GOOGLE_PROJECT_ID hashicorp/packer:full-1.11.0 validate scylla.json
Unable to find image 'hashicorp/packer:full-1.11.0' locally
full-1.11.0: Pulling from hashicorp/packer
d25f557d7f31: Pull complete 
9781459adf19: Pull complete 
03c043e2e43d: Pull complete 
895d5744c142: Pull complete 
65ae4a874074: Pull complete 
d7b348b3f6b4: Pull complete 
Digest: sha256:5546bddad4ae06221932b0819c10b503a5caf0946dd8d6552956723581f6a308
Status: Downloaded newer image for hashicorp/packer:full-1.11.0
Error: Failed to prepare build: "aws"

4 error(s) occurred:

* ami_name must be specified
* ami_name must be between 3 and 128 characters long
* An ssh_username must be specified
  Note: some builders used to default ssh_username to "root".
* either instance_type or spot_instance_types must be specified

Error: Failed to prepare build: "azure"

1 error(s) occurred:

* A capture_container_name, managed_image_name or
shared_image_gallery_destination must be specified

Error: Failed to prepare build: "gce"

4 error(s) occurred:

* An ssh_username must be specified
  Note: some builders used to default ssh_username to "root".
* a project_id must be specified
* a source_image or source_image_family must be specified
* a zone must be specified

I will work on it to see if i can get it working.

yaronkaikov commented 1 week ago

@nywilken Sorry the reason for those errors is that we are using parameters and when I was running it I didn't provide them.

once i provide them i get

[yaronkaikov@london]~/git/scylla-machine-image/packer (master)$ docker run -v `pwd`:/workspace -w /workspace -eGOOGLE_PROJECT_ID=$GOOGLE_PROJECT_ID hashicorp/packer:full-1.11.0 validate scylla.json
The configuration is valid.

nywilken commented 1 week ago

@nywilken Sorry the reason for those errors is that we are using parameters and when I was running it I didn't provide them.

once i provide them i get
[yaronkaikov@london]~/git/scylla-machine-image/packer (master)$ docker run -v `pwd`:/workspace -w /workspace -eGOOGLE_PROJECT_ID=$GOOGLE_PROJECT_ID hashicorp/packer:full-1.11.0 validate scylla.json
The configuration is valid.

@yaronkaikov no worries, thank you for the quick follow-up. I'm glad you were able to get this squared away. I appreciate you opening an issue to start. Packer 1.11.0 changed how Packer works with plugins so it is great to know if folks run into any issues.

Keep building and don't hesitate to reach out if you run into any issue that seems like a bug.

I'm going to close this issue. But if anything should change drop me a comment and I will gladly reopen it.

yaronkaikov commented 1 week ago

@nywilken Sorry the reason for those errors is that we are using parameters and when I was running it I didn't provide them. once i provide them i get
[yaronkaikov@london]~/git/scylla-machine-image/packer (master)$ docker run -v `pwd`:/workspace -w /workspace -eGOOGLE_PROJECT_ID=$GOOGLE_PROJECT_ID hashicorp/packer:full-1.11.0 validate scylla.json
The configuration is valid.
@yaronkaikov no worries, thank you for the quick follow-up. I'm glad you were able to get this squared away. I appreciate you opening an issue to start. Packer 1.11.0 changed how Packer works with plugins so it is great to know if folks run into any issues.

Keep building and don't hesitate to reach out if you run into any issue that seems like a bug.

I'm going to close this issue. But if anything should change drop me a comment and I will gladly reopen it.

@nywilken I meant that the configuration is valid, but the failure remains the same :-(

yaronkaikov commented 1 week ago

@nywilken some more information i got from the log

2024/06/25 11:10:32 [INFO] Packer version: 1.11.0 [go1.21.10 linux amd64]
2024/06/25 11:10:32 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2024/06/25 11:10:32 [INFO] PACKER_CONFIG env var set; attempting to open config file: /tmp/.packerconfig
2024/06/25 11:10:32 [WARN] Config file doesn't exist: /tmp/.packerconfig
2024/06/25 11:10:32 [INFO] Setting cache directory: /tmp/.cache/packer
e: Running in background, not using a TTY
2024/06/25 11:10:32 [TRACE] listing potential installations for <nil> that match "". plugingetter.ListInstallationsOptions{PluginDirectory:"/tmp/.config/packer/plugins", BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"linux", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc00073a200)}}, ReleasesOnly:false}}
2024/06/25 11:10:32 Directory "/tmp/.config/packer/plugins" does not exist, the plugin likely isn't installed locally yet.
2024/06/25 11:10:32 ui error: Error: Failed to initialize build "gce"

The builder googlecompute is unknown by Packer, and is likely part of a plugin
that is not installed.
You may find the needed plugin along with installation instructions documented
on the Packer integrations page.

https://developer.hashicorp.com/packer/integrations?filter=googlecompute

2024/06/25 11:10:32 [INFO] (telemetry) Finalizing.
2024/06/25 11:10:32 waiting for all plugin processes to complete...

Why is it looking under /tmp instead /root ?

yaronkaikov commented 1 week ago

Oh, i am passing this with my container, i will check it then , and update

nywilken commented 1 week ago

Hi @yaronkaikov apologies I thought it was resolved since the configuration validated properly. The plugins are loaded during validate hence why I thought it was resolved.

But it looks like you figure it out now. Please let me know if you need more help and if I should reopen this issue.

yaronkaikov commented 2 days ago

@nywilken Just a quick update, managed to solve it by adding PACKER_PLUGIN_PATH to the env with the correct location. Thanks for your support

hashicorp / packer