dkirrane commented 6 years ago

Is there any vsphere-iso example for creating a vm from an ISO and Kickstart file.

michaelhajjar commented 6 years ago

I am currently using the following packer json file for creating centos7 template on vsphere version 6 (with no boot_cmd option, because the API does not support it).

{
  "variables": {
    "vcenter_host": "server",
    "vcenter_user": "user",
    "vcenter_password": "",
    "ssh_user": "user",
    "ssh_password": "password",
    "dc": "domain",
    "cluster": "cluster",
    "network": "network",
    "storage": "datastore",
    "template_dir": "dir",
    "iso_path": "iso",
    "ks_file": "kickstart/ks.cfg",
    "cpu_cores": "2",
    "ram_mb": "4096",
    "disk_size_gb": "50"
  },

  "builders": [
    {
      "type": "vsphere-iso",

      "vcenter_server":      "{{ user `vcenter_host` }}",
      "username":            "{{ user `vcenter_user` }}",
      "password":            "{{ user `vcenter_password` }}",
      "insecure_connection": "true",
      "ssh_username": "{{ user `ssh_user` }}",
      "ssh_password": "{{ user `ssh_password` }}",

      "datacenter": "{{ user `dc` }}",
      "cluster": "{{ user `cluster` }}",
      "network": "{{ user `network` }}",
      "network_card": "vmxnet3",
      "datastore": "{{ user `storage` }}",
      "vm_name": "centos7_base_{{isotime \"2006-01-02\"}}",
      "guest_os_type": "rhel7_64Guest",

      "CPUs":             "{{ user `cpu_cores` }}",
      "RAM":              "{{ user `ram_mb` }}",
      "RAM_reserve_all": true,

      "disk_controller_type":  "pvscsi",
      "disk_size":        "{{ user `disk_size_gb` }}",
      "disk_thin_provisioned": true,
      "folder": "{{ user `template_dir` }}",

      "iso_paths": [
        "{{ user `iso_path` }}"
      ],
      "floppy_files": [
        "{{ user `ks_file` }}"
      ],
      "boot_order": "disk,cdrom",
      "convert_to_template": true
    }
  ],

  "provisioners": [
    {
      "type": "shell",
      "inline": ["rm -rf /etc/yum.repos.d/*"],
      "execute_command": "echo 'root' | {{.Vars}} sudo -S -E bash '{{.Path}}'",
      "pause_before": "10s"
    },

    { 
      "type": "file",
      "source": "repos/",
      "destination": "/etc/yum.repos.d"
    },

    {
      "type": "shell",
      "execute_command": "echo 'root' | {{.Vars}} sudo -S -E bash '{{.Path}}'",
      "script": "scripts/centos7_cleanup.sh",
      "expect_disconnect": "true"
    }
  ]
}

You can also use the boot_cmd option if you have vsphere 6.5, otherwise you can edit the centos7 iso and edit the isolinux.cfg to point to the floppy disk(kickstart file) in this setup.

dkirrane commented 6 years ago

I can create a kickstart ISO that can load automatically without editing the Centos ISO. Just need to add the ks.cfg into a storage volume named OEMDRV mkisofs -V OEMDRV -o kickstart.iso ks.cfg

Can an extra CD-ROM device be added for the kickstart ISO? And can Packer remove all CD-ROM devices once install completes?

Must look at the boot_cmd option for 6.5. Could probably just pass boot option inst.ks=<location of ks.cfg>

sudomateo commented 6 years ago

@dkirrane I attempted to use Packer to create a CentOS image with a kickstart.iso created with mkisofs -V OEMDRV -o kickstart.iso ks.cfg successfully. The trick is to attach the kickstart.iso file as a secondary CD/DVD drive using the iso_paths parameter.

My template:

{
  "description": "CentOS 7 Template",
  "variables": {
    "vsphere_username": "{{ env `VSPHERE_USERNAME` }}",
    "vsphere_password": "{{ env `VSPHERE_PASSWORD` }}",
    "vm_name": "centos-7-base-x86_64-1708"
    "vcenter_server": "vcenter.example.com",
    "datacenter": "DATACENTER01",
    "datastore": "DATASTORE01",
    "folder": "Templates/Packer",
    "cluster": "CLUSTER01",
    "iso_paths": "[DATASTORE01] ISO/CentOS-7-x86_64-NetInstall-1708.iso,[DATASTORE01] ISO/kickstart.iso",
  "network": "VM Network"
  },
  "builders": [
    {
      "type": "vsphere-iso",
      "vm_name": "{{ user `vm_name` }}-{{ isotime \"2006-01-02\" }}",
      "vcenter_server": "{{ user `vcenter_server` }}",
      "username": "{{ user `vsphere_username` }}",
      "password": "{{ user `vsphere_password` }}",
      "insecure_connection": "true",
      "datacenter": "{{ user `datacenter` }}",
      "cluster": "{{ user `cluster` }}",
      "folder": "{{ user `folder` }}",
      "datastore": "{{ user `datastore` }}",
      "boot_wait": "30s",
      "boot_order": "disk,cdrom",
      "iso_paths": "{{ user `iso_paths` }}",
      "ssh_username": "root",
      "ssh_password": "packer",
      "CPUs": 1,
      "RAM": 1024,
      "RAM_reserve_all": true,
      "disk_controller_type": "pvscsi",
      "disk_size": 16384,
      "disk_thin_provisioned": true,
      "network": "{{ user `network` }}",
      "network_card": "vmxnet3",
      "guest_os_type": "rhel7_64Guest",
      "convert_to_template": true
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "only": ["vsphere-iso"],
      "script": "<PATH_TO_SCRIPT>"
    }
  ]
}

ideologysec commented 6 years ago

Packer does IIRC remove all CD-ROM devices when the build is finished, or at least this builder does, but only if you don't convert it to a template.

dkirrane commented 6 years ago

@sudomateo I've used the following steps with govc

My kickstart ks.cfg file Note the cmdline, poweroff, passed in network info {{}}, services enabled and post install sections were required for me. The rest is pretty standard from a normal manual install.
```
#version=DEVEL
```

Use non-interactive command line mode

cmdline eula --agreed logging --level=info selinux --disabled firewall --disabled poweroff

System authorization information

auth --enableshadow --passalgo=sha512

Use CDROM installation media

cdrom

Run the Setup Agent on first boot

firstboot --disabled

Keyboard layouts

keyboard --vckeymap=gb --xlayouts='gb'

System language

lang en_GB.UTF-8

Network information

network --onboot=no --device=link --bootproto=static --ip={{ ip }} --netmask={{ netmask }} --gateway={{ gateway }} --nameserver={{ dns_servers }} --hostname={{ hostname.domain }} --noipv6

Root password

rootpw --iscrypted XYZZZZZZZZZZZZZZZZ

System services

services --enabled=NetworkManager,sshd,chronyd

System timezone

timezone Europe/London --isUtc

System bootloader configuration

bootloader --location=mbr --boot-drive=sda autopart --type=lvm

Partition clearing information

clearpart --none --initlabel

%packages @^minimal @core chrony

%end

%addon com_redhat_kdump --disable --reserve-mb='auto'

%end

%anaconda pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --notempty pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty %end

Post Install

%post --log=/root/ks-post.log yum install -y epel-release yum install -y open-vm-tools yum install -y perl %end



2. Create kickstart iso
`mkisofs -V OEMDRV -o kickstart.iso ks.cfg`

3. Upload CentOS iso. (As downloaded. No changes needed)
`govc datastore.upload CentOS-7-x86_64-Minimal-1708.iso isos/CentoOS.iso`

4. Upload kickstart iso
`govc datastore.upload kickstart.iso isos/kickstart.iso`

5. Create VM (called `MyVM`) from CentOS ISO (do not power on, add disk, ram cpu, vlan settings also as required)
`govc vm.create -g=centos64Guest -on=false -iso=isos/CentoOS.iso -iso-datastore=DataStoreXYZ MyVM`

6. Add CD-ROM device kickstart iso
`govc device.cdrom.add -vm MyVM`

7. Insert kickstart iso into new CD-ROM
`govc device.cdrom.insert -vm MyVM -device {{kickstart_cdrom_device_num_from_step_6}} isos/kickstart.iso`

8. Set boot order
`govc device.boot -vm MyVM -delay 1000 -order cdrom,ethernet,disk`

9. Power on VM
`govc vm.power -on MyVM`

10. Wait for kickstart to complete. At this stage you can follow the automated kickstart install from the vSphere client console
To wait I retry this command until PowerState == `poweredOff`
`govc vm.info -json MyVM jq -r .VirtualMachines[].Runtime.PowerState`

11. Get all CD-ROM devices numbers
`govc device.info -json -vm MyVM cdrom-*`

12. Remove all CD-ROM devices from VM
`govc device.remove -vm MyVM {{kickstart_cdrom_device_num_from_step_11}}`
`govc device.remove -vm MyVM {{centos_cdrom_device_num_from_step_11}}`

13. Enable VM UUID
`govc vm.change -vm MyVM -e="disk.enableUUID=1"`

14. Convert to template
`govc vm.markastemplate MyVM`

MacFlurry commented 5 years ago

Hello,

I followed all the instructions but I still have a problem here: screen shot here's my kickstart and packer file. if someone with a keen eye can tell me where I was wrong:

vphere 6.5 packer json file:

{
    "variables": {
      "vcenter_host": "xxx",
      "vcenter_user": "xxxx",
      "vcenter_password": "xxxx",
      "ssh_user": "xxxx",
      "ssh_password": "xxxxx",
      "dc": "xxxx",
      "cluster": "xxx",
      "network": "xxx",
      "storage": "xxx",
      "ks_file": "ks.cfg",
      "cpu_cores": "1",
      "ram_mb": "1024",
      "disk_size_gb": "30"
    },

    "builders": [
      {
        "type": "vsphere-iso",

        "vcenter_server":      "{{ user `vcenter_host` }}",
        "username":            "{{ user `vcenter_user` }}",
        "password":            "{{ user `vcenter_password` }}",
        "insecure_connection": "true",
        "ssh_username": "{{ user `ssh_user` }}",
        "ssh_password": "{{ user `ssh_password` }}",

        "datacenter": "{{ user `dc` }}",
        "cluster": "{{ user `cluster` }}",
        "network": "{{ user `network` }}",
        "network_card": "vmxnet3",
        "datastore": "{{ user `storage` }}",
        "vm_name": "centos7_test",
        "guest_os_type": "rhel7_64Guest",

        "CPUs":             "{{ user `cpu_cores` }}",
        "RAM":              "{{ user `ram_mb` }}",
        "RAM_reserve_all": true,

        "disk_controller_type":  "pvscsi",
        "disk_size":        "{{ user `disk_size_gb` }}",
        "disk_thin_provisioned": true,

        "iso_paths": [
            "[xxxxxxxxx] upload/CentOS-7-x86_64-Minimal-1810.iso", 
            "[xxxxxxxxx] upload/kickstart.iso"
        ],

        "boot_order": "disk,cdrom"
      }
    ],

    "provisioners": [
      {
        "type": "shell",
        "inline": [ "ls /"],
        "execute_command": "echo 'root' | {{.Vars}} sudo -S -E bash '{{.Path}}'",
        "pause_before": "10s"
      }
    ]
  }

kickstart file ks.cfg:

# Basic Configuration
lang en_US
keyboard fr
cmdline
eula --agreed
timezone Europe/Paris --isUtc
rootpw $1$GDX6u2Ls$AZ4Eq5d07tPglQKSYL5j4/ --iscrypted
#platform x86, AMD64, or Intel EM64T
reboot
# Installation
text
cdrom
# BootLoader
bootloader --location=mbr --timeout=10 --append="rhgb quiet crashkernel=auto"
zerombr
# Partition
clearpart --all --initlabel --drives=sda
autopart --type=lvm
# Auth
auth --passalgo=sha512 --useshadow
# Disable selinux & Fw
selinux --disabled
firewall --disabled
# No GUI
skipx
firstboot --disable
# Post installation
%post --nochroot --interpreter=/bin/bash
yum install -y open-vm-tools
systemctl enable vmtoolsd
systemctl start vmtoolsd
yum install -y vim
yum install -y redhat-lsb-core
yum update -y
%end
# Packages
%packages
@^minimal
@core
-fprintd-pam
-rfkill
-rhnsd
-nano
-usbutils
-plymouth
%end

Thanks for your help.

jason-azze commented 5 years ago

Thanks for providing some examples, friends. You saved me a ton of time while getting CentOS 7 working with vSphere 5.5.

I also had to get CentOS 6 working. Here are a few things I learned.

CentOS 6 doesn't "know" about the OEMDRV label. So mounting a kickstart.iso alongside your installation media so it can automagically be found won't work. Instead I had to make a customized modification of CentOS-6.10-x86_64-bin-DVD1.iso by following the instructions in section "32.9.1. Creating Kickstart Boot Media" in the RHEL 6 installation guide.
But now you have a new problem -- epel-release and open-vm-tools are not available on DVD1. Fine. You can just yum install them in the %post section of your ks.cfg. Except, /etc/resolv.conf hasn't been set up yet when the %post step runs. So yum doesn't work. After several hours of fiddling with different ideas (including constructing my own, custom "Everything" iso), I decided to construct a temporary /etc/resolv.conf during %post and then run the package installs.

%post --log=/root/ks-post.log
echo "search myinternalnamespace.com" >> /etc/resolv.conf
echo "nameserver 8.8.8.8" >> /etc/resolv.conf
yum install -y epel-release
yum install -y open-vm-tools
service vmtoolsd start
chkconfig vmtoolsd on
%end

Another CentOS 6 quirk if you are copy-and-pasting the examples here.

CentOS 6 kickstart doesn't understand or need eula --agreed

And extra thanks to @sudomateo for the example up there ^^^. You saved me a ton of time with CentOS 7. But watch our for that "disk_size": 16,. That only gives you 16MB. It had me scratching my head for a bit. :-)

MacFlurry commented 5 years ago

That only gives you 16MB. It had me scratching my head for a bit. :-)

Haha , indeed. that also turned me crazy until I figured out that it was in MB. 😂😂

MacFlurry commented 5 years ago

for vsphere 6.5 I made Packer and Kickstart successfully work for template VM.

sudomateo commented 5 years ago

And extra thanks to @sudomateo for the example up there ^^^. You saved me a ton of time with CentOS 7. But watch our for that "disk_size": 16,. That only gives you 16MB. It had me scratching my head for a bit. :-)

You're very welcome! Haha at the time of my writing, the disk_size parameter was in gigabytes. Sorry for the confusion! I'll edit my response above.

@jason-azze @MacFlurry

luckyinva commented 4 years ago

for vsphere 6.5 I made Packer and Kickstart successfully work for template VM.

@MacFlurry Can you post the code that you go to work?

MacFlurry commented 4 years ago

for vsphere 6.5 I made Packer and Kickstart successfully work for template VM.

@MacFlurry Can you post the code that you go to work?

oh sorry, I've been pretty busy lately. Do you still need the code?

MacFlurry commented 4 years ago

Just in case someone want it:

{
    "variables": {
      "vcenter_host": "{{env `VCENTER_HOST` }}",
      "vcenter_user": "admin_name",
      "vcenter_password": "{{env `VCENTER_PASSWORD` }}",
      "ssh_host": "{{env `SSH_HOST` }}",
      "ssh_port": "22",
      "ssh_user": "root",
      "ssh_password": "put_password_here",
      "vm_name": "vmname_goes_here",
      "dc": "{{env `VCENTER_DATACENTER` }}",
      "cluster": "{{env `VCENTER_CLUSTER` }}",
      "network": "{{env `VCENTER_NETWORK` }}",
      "storage": "{{env `VCENTER_STORAGE` }}",
      "ks_file": "ks.cfg",
      "cpu_cores": "1",
      "ram_mb": "1024",
      "disk_size_mb": "14336"
    },

    "builders": [
      {
        "type": "vsphere-iso",

        "vcenter_server":      "{{ user `vcenter_host` }}",
        "username":            "{{ user `vcenter_user` }}",
        "password":            "{{ user `vcenter_password` }}",
        "insecure_connection": "true",
        "ssh_username": "{{ user `ssh_user` }}",
        "ssh_password": "{{ user `ssh_password` }}",
        "ssh_timeout": "600s",
        "ssh_port": "{{ user `ssh_port` }}",
        "ssh_host": "{{ user `ssh_host` }}",

        "datacenter": "{{ user `dc` }}",
        "cluster": "{{ user `cluster` }}",
        "network": "{{ user `network` }}",
        "network_card": "vmxnet3",
        "datastore": "{{ user `storage` }}",
        "vm_name": "{{ user `vm_name` }}",
        "guest_os_type": "centos7_64Guest",

        "CPUs":             "{{ user `cpu_cores` }}",
        "RAM":              "{{ user `ram_mb` }}",
        "RAM_reserve_all": true,
        "convert_to_template": true,

        "disk_controller_type":  "pvscsi",
        "disk_size":        "{{ user `disk_size_mb` }}",
        "disk_thin_provisioned": true,

        "iso_paths": [
            "[datastore_name] directory/centos_7_original.iso"
        ],

        "floppy_files": [
          "{{template_dir}}/ks.cfg"
        ],
        "boot_wait": "10s",
        "boot_command": "<esc> <wait> linux inst.text inst.ks=hd:fd0:/ks.cfg <enter>"
       }
    ],

    "provisioners": [
      {
        "type": "shell",
        "inline": [ "ls /"],
        "execute_command": "echo 'root' | {{.Vars}} sudo -S -E bash '{{.Path}}'",
        "pause_before": "10s"
      }
    ]
  }

soakes commented 4 years ago

@MacFlurry Thanks very much for the snippet, that saves me a little time :)

MacFlurry commented 4 years ago

You're welcomed :-)

kbcz1989 commented 4 years ago

inst.ks=hd:fd0:/ks.cfg

This will not work on CentOS 8+. I had to revert to http..

ideologysec commented 4 years ago

@kbcz1989 confirmed. I haven't had a chance to dig into why, but I think either kickstart dropped floppy support or the default vmware/floppy driver isn't included anymore.

allaboutopensource commented 4 years ago

i am able to spin the vm in vmware esxi 6.7 but the build is failing at "Build 'vsphere-iso' errored: Timeout waiting for SSH"

getting an error

“2020/04/10 04:40:05 packer-builder-vsphere-iso plugin: [DEBUG] TCP connection to SSH ip/port failed: dial tcp 10.0.68.221:22: i/o timeout”

i have given the same ssh_username, password what is there in the kickstart file but still the same . ssh_timeout is 5min .

my kickstart file is as below:

Basic Configuration

lang en_US keyboard fr cmdline eula --agreed timezone Europe/Paris --isUtc network --onboot yes --device ens192 --bootproto static --ip=10.0.68.221 --netmask=255.255.255.0 --gateway= 10.0.68.1 --nameserver=172.30.0.150 --hostname=centos.local --noipv6 services --enabled=iptables,rsyslog,sshd,NetworkManager user --groups=wheel --homedir=/home/itops --name=itops --iscrypted --password=$1$c2dnIt5L$UsKvFG/u0Mzh7VBHfOR3c0 rootpw --iscrypted $1$c2dnIt5L$UsKvFG/u0Mzh7VBHfOR3c0

platform x86, AMD64, or Intel EM64T

reboot

Installation

text cdrom

BootLoader

bootloader --location=mbr --timeout=10 --append="rhgb quiet crashkernel=auto" zerombr sshpw --username root $1$c2dnIt5L$UsKvFG/u0Mzh7VBHfOR3c0 --iscrypted

Partition

clearpart --all --initlabel --drives=sda autopart --type=lvm

Auth

auth --passalgo=sha512 --useshadow

Disable selinux & Fw

selinux --disabled firewall --disabled

No GUI

skipx firstboot --disable

Post installation

%post --nochroot --interpreter=/bin/bash yum install -y vim mv /etc/ssh/sshd_config.anaconda /etc/ssh/sshd_config systemctl start sshd %end

Packages

%packages @^minimal @core -usbutils %end reboot --eject

kbcz1989 commented 4 years ago

@allaboutopensource so, installation finishes or not? Do you see the IP address in VSphere? Can you connect manually to that IP from packer IP?

allaboutopensource commented 4 years ago

i can see the IP address in the vsphere with the login page showing up but i cannot ping the ip address and nor tenlet to 22 port.

kbcz1989 commented 4 years ago

That sounds like a network problem. If you install the machine manually and set the same ip, can you then ping/telnet from packer machine?

allaboutopensource commented 4 years ago

i am able to create the vm with manually assigned ip address but it just taking more then 5 minutes to activate the network on it.

one more thing is that when i try to login to the vmware vm console manually with the username and password it says incorrect login.

jetbrains-infra / packer-builder-vsphere

vsphere-iso example for CentOS #85

Use non-interactive command line mode

System authorization information

Use CDROM installation media

Run the Setup Agent on first boot

Keyboard layouts

System language

Network information

Root password

System services

System timezone

System bootloader configuration

Partition clearing information

Post Install

Basic Configuration

platform x86, AMD64, or Intel EM64T

Installation

BootLoader

Partition

Auth

Disable selinux & Fw

No GUI

Post installation

Packages