search

hashicorp / packer-plugin-vsphere

Packer plugin for VMware vSphere Builder
https://www.packer.io/docs/builders/vsphere
Mozilla Public License 2.0
94 stars 91 forks source link

vsphere-clone: can't set static IP when using cloud-init image #335

Closed jsquyres closed 4 months ago

jsquyres commented 9 months ago

Overview of the Issue

I am using vsphere-clone to clone an OVA of Alma 9 that was just uploaded to my vsphere. I cannot figure out how to set a static IP address in the resulting packer-instantiated VM:

  1. If I use vapp --> properties --> user-data to specify cloud init data, it seems to execute (mostly) properly, but somehow the network cloud init stanza gets wholly ignored and the VM tries to DHCP (in my environment, the DHCP will fail, and the VM will come up without networking setup).
  2. If I use customize and try to set a fixed IP address via network_interfaces, the VM comes up with the network device present but disconnected in vsphere. Since the device is disconnected, no networking gets setup in the VM.

Reproduction Steps

There's 2 cases I'm unable to make work properly:

Using vapp --> properties --> user-data (i.e., clout init)
  1. Make a packer builder config file with vapp --> properties --> user-data with base64-encoded cloud init data
  2. Have a basic runcmd stanza in user-data (echo hello world)
  3. Have a network stanza in user-data that assigns a static IP
  4. Run packer with vsphere-clone, cloning a freshly-created Alma 9 OVA
  5. See the VM try to DHCP during bootup, and timeout after 60 seconds (I have no DHCP server in my environment)
  6. Watch cloud init run
  7. See the echo hello world run from cloud init
  8. See that the fixed IP address from my cloud init was not set
Using customize
  1. Make a packer builder config file with customize that sets a fixed IP address (via network_interface)
  2. Run packer with vsphere-clone, cloning a freshly-created Alma 9 OVA
  3. See the VM skip over the network setup quickly during bootup
  4. In Vsphere, see that the network device is present in the VM, but disconnected

Packer Version

$ packer --version
1.9.4

Plugin Version and Builders

Please provide the plugin version.

1.2.3

Please select the builder.

VMware vSphere Version

Please provide the VMware vSphere version.

7.0.3.01700

Guest Operating System

Alma 9 (latest).

Simplified Packer Buildfile

There's 2 cases I'm unable to make work properly:

Using vapp --> properties --> user-data (i.e., clout init)

This case executes all the cloud init stuff in my Alma 9 VM properly, except that my network stanza is wholly skipped. Instead, I can see in the cloud-init logs that it tried to DHCP. Does the vmware-clone plugin somehow substitute in its own network stanza to cloud init that overrides my network stanza (and tells cloud init to DHCP)?

{
    "builders": [
        {
            "type": "vsphere-clone",
            ...
            "vapp": {
                "properties": {
                    "user-data": "...the b64 of my cloud init user-data",
                }
            }
        }
    ]
}

My cloud init user-data is quite bare bones:

#cloud-config                                                                                                            
users:
  - name: testuser
    sudo: ALL=(ALL) NOPASSWD:ALL
    lock_passwd: false
    passwd: ...hashed password that enables me to login on the vsphere console...

runcmd:
  - echo hello world

network:
  version: 2
  ethernets:
    ens192:
        ....etc.

The user is created properly (and I can login on the console), and the "hello world" is emitted in the logs. So all that ran correctly.

But per above, my network stanza is ignored and somehow cloud init tries to DHCP, instead.

Using customize
{
    "builders": [
        {
            "type": "vsphere-clone",
            ...
            "customize" : {
                "linux_options": {
                    "host_name": "alma9-test",
                    "domain": "example.com"
                },
                "network_interface": {
                    "ipv4_address": "10.10.10.10",
                    "ipv4_netmask": "24"
                },
                "ipv4_gateway": "10.10.10.1"
            }
        }
    ]
}

In this case, the VM boots up with a network device present, but disconnected. Hence, whatever networking is setup in the VM initialization doesn't occur. If I go into vsphere and click the "Connected" checkbox next to the network device, it connects to the VM properly, but whatever network setup is present (if any?) doesn't fire, and no fixed IP address is set.

Operating System and Environment Details

I am using an OVA that we created from the latest Alma 9 distribution ISO. We actually use packer to launch the latest Alma 9 distribution ISO with a minimal kickstart file, and then have packer generate an OVA from that. It is this OVA that I am attempting to use in a separate execution of packer with vsphere-clone.

Log Fragments and crash.log Files

2023/12/01 23:08:33 [INFO] Packer version: 1.9.4 [go1.20.7 linux amd64]
2023/12/01 23:08:33 [TRACE] discovering plugins in /usr/bin
2023/12/01 23:08:33 [TRACE] discovering plugins in .
2023/12/01 23:08:33 [TRACE] discovering plugins in /home/jsquyres/.config/packer/plugins
2023/12/01 23:08:33 [INFO] Discovered potential plugin: ansible = /home/jsquyres/.config/packer/plugins/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64
2023/12/01 23:08:33 [INFO] Discovered potential plugin: vsphere = /home/jsquyres/.config/packer/plugins/github.com/hashicorp/vsphere/packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64
2023/12/01 23:08:33 found external [-packer-default-plugin-name- local] provisioner from ansible plugin
2023/12/01 23:08:33 [INFO] found external [clone iso supervisor] builders from vsphere plugin
2023/12/01 23:08:33 [INFO] found external [-packer-default-plugin-name- template] post-processors from vsphere plugin
2023/12/01 23:08:33 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/12/01 23:08:33 [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/jsquyres/.packerconfig
2023/12/01 23:08:33 [WARN] Config file doesn't exist: /home/jsquyres/.packerconfig
2023/12/01 23:08:33 [INFO] Setting cache directory: /home/jsquyres/.cache/packer
2023/12/01 23:08:33 [INFO] Starting external plugin /home/jsquyres/.config/packer/plugins/github.com/hashicorp/vsphere/packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 start builder clone
2023/12/01 23:08:33 Starting plugin: /home/jsquyres/.config/packer/plugins/github.com/hashicorp/vsphere/packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 []string{"/home/jsquyres/.config/packer/plugins/github.com/hashicorp/vsphere/packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64", "start", "builder", "clone"}
2023/12/01 23:08:33 Waiting for RPC address for: /home/jsquyres/.config/packer/plugins/github.com/hashicorp/vsphere/packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64
2023/12/01 23:08:33 packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 plugin: 2023/12/01 23:08:33 Plugin address: unix /tmp/packer-plugin3426207495
2023/12/01 23:08:33 Received unix RPC address for /home/jsquyres/.config/packer/plugins/github.com/hashicorp/vsphere/packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64: addr is /tmp/packer-plugin3426207495
2023/12/01 23:08:33 packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 plugin: 2023/12/01 23:08:33 Waiting for connection...
2023/12/01 23:08:33 packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 plugin: 2023/12/01 23:08:33 Serving a plugin connection...
2023/12/01 23:08:33 packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 plugin: 2023/12/01 23:08:33 [TRACE] starting builder clone
2023/12/01 23:08:33 [INFO] Starting internal plugin packer-provisioner-shell
2023/12/01 23:08:33 Starting plugin: /usr/bin/packer []string{"/usr/bin/packer", "plugin", "packer-provisioner-shell"}
2023/12/01 23:08:33 Waiting for RPC address for: /usr/bin/packer
2023/12/01 23:08:33 Received unix RPC address for /usr/bin/packer: addr is /tmp/packer-plugin2316675395
2023/12/01 23:08:33 packer-provisioner-shell plugin: [INFO] Packer version: 1.9.4 [go1.20.7 linux amd64]
2023/12/01 23:08:33 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/12/01 23:08:33 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/jsquyres/.packerconfig
2023/12/01 23:08:33 packer-provisioner-shell plugin: [WARN] Config file doesn't exist: /home/jsquyres/.packerconfig
2023/12/01 23:08:33 packer-provisioner-shell plugin: [INFO] Setting cache directory: /home/jsquyres/.cache/packer
2023/12/01 23:08:33 packer-provisioner-shell plugin: args: []string{"packer-provisioner-shell"}
2023/12/01 23:08:33 packer-provisioner-shell plugin: Plugin address: unix /tmp/packer-plugin2316675395
2023/12/01 23:08:33 packer-provisioner-shell plugin: Waiting for connection...
2023/12/01 23:08:33 packer-provisioner-shell plugin: Serving a plugin connection...
2023/12/01 23:08:33 [INFO] Starting internal plugin packer-post-processor-shell-local
2023/12/01 23:08:33 Starting plugin: /usr/bin/packer []string{"/usr/bin/packer", "plugin", "packer-post-processor-shell-local"}
2023/12/01 23:08:33 Waiting for RPC address for: /usr/bin/packer
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: [INFO] Packer version: 1.9.4 [go1.20.7 linux amd64]
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/jsquyres/.packerconfig
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: [WARN] Config file doesn't exist: /home/jsquyres/.packerconfig
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: [INFO] Setting cache directory: /home/jsquyres/.cache/packer
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: args: []string{"packer-post-processor-shell-local"}
2023/12/01 23:08:33 Received unix RPC address for /usr/bin/packer: addr is /tmp/packer-plugin2087816507
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: Plugin address: unix /tmp/packer-plugin2087816507
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: Waiting for connection...
2023/12/01 23:08:33 packer-post-processor-shell-local plugin: Serving a plugin connection...
2023/12/01 23:08:33 Preparing build: vsphere-clone
vsphere-clone: output will be in this color.

2023/12/01 23:08:33 Build debug mode: false
2023/12/01 23:08:33 Force build: false
2023/12/01 23:08:33 On error: 
2023/12/01 23:08:33 Waiting on builds to complete...
2023/12/01 23:08:33 Starting build run: vsphere-clone
2023/12/01 23:08:33 Running builder: vsphere-clone
2023/12/01 23:08:33 [INFO] (telemetry) Starting builder vsphere-clone
2023/12/01 23:08:33 packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 plugin: 2023/12/01 23:08:33 No CD files specified. CD disk will not be made.
==> vsphere-clone: Cloning VM...
2023/12/01 23:08:33 packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 plugin: 2023/12/01 23:08:33 fetching properties for VM "/...sanitized.../TEMPLATE-test-alma9"
==> vsphere-clone: Customizing hardware...
==> vsphere-clone: Mounting ISO images...
==> vsphere-clone: Adding configuration parameters...
==> vsphere-clone: Customizing VM...
2023/12/01 23:08:37 packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 plugin: 2023/12/01 23:08:37 No floppy files specified. Floppy disk will not be made.
==> vsphere-clone: Power on VM...
2023/12/01 23:08:39 packer-plugin-vsphere_v1.2.3_x5.0_linux_amd64 plugin: 2023/12/01 23:08:39 [INFO] Waiting for IP, up to total timeout: 30m0s, settle timeout: 5s
==> vsphere-clone: Waiting for IP...
...stuck here...
jsquyres commented 8 months ago

Ping. Any thoughts on this? Thanks!

jsquyres commented 6 months ago

It's been a few months. Any thoughts on this issue?

tenthirtyam commented 4 months ago

Hi there! Based on the details provided it's unknown...:

Ryan Johnson Distinguished Engineer, VMware by Broadcom

tenthirtyam commented 4 months ago

Hello! 👋

This issue has not received any update from the OP in the last 7 days.

Please feel free to comment and/or reopen this issue if the issue persists.