knife vsphere vm clone command creates its own vlan standard port group in vCenter, instead of joining the specified --cvlan distrubuted port group

Zorrom commented 6 years ago

Versions:

Version of knife-vsphere: 2.0.1
Version of chef: 12.19.36

Platform Details

Version of vSphere: 6.5.0
Version of ESXi: 6.5.0

Scenario:

I am trying to spin a vm using the following command

knife vsphere vm clone testVM --dest-folder Chef_Test --template TEST_TMPT --start --cvlan vlan01 --datastore my_datastore --cspec winspec

The above command spins the vm with my customization template options but instead of joining the vlan01 Distributed Port Group(dvSwitch3), Chef creates a completely new vlan01 Standard Network Port Group for itself on the vCenter and adds the vm there.

So as means of this the VM is joined to a completely different Vlan that never existed in the vCenter itself. This causes the VM network adapter settings to stay disconnected and the network connections are failed in short.

Please paste the command and output (run with `-VV`). Don't forget to X out any passwords or sensitive information:

[root@chefworkstation .chef]# knife vsphere vm clone testVM --dest-folder Chef_Test --template TEST_CHEF_TMPT --start --cvlan VLAN01 --datastore my_datastore --cspec Windows_spec -VV
Cloning template TEST_CHEF_TMPT to new VM testVM
VirtualMachineCloneSpec(
  config: VirtualMachineConfigSpec(
    deviceChange: [VirtualDeviceConfigSpec(
       device: VirtualE1000e(
         addressType: "assigned",
         backing: VirtualEthernetCardNetworkBackingInfo(
           deviceName: "VLAN01",
           network: DistributedVirtualPortgroup("dvportgroup-142")
         ),
         connectable: VirtualDeviceConnectInfo(
           allowGuestControl: true,
           connected: false,
           startConnected: true,
           status: "untried"
         ),
         controllerKey: 100,
         deviceInfo: Description(
           label: "Network adapter 1",
           summary: "my_PortGroup"
         ),
         key: 4000,
         macAddress: nil,
         resourceAllocation: VirtualEthernetCardResourceAllocation(
           limit: -1,
           reservation: 0,
           share: SharesInfo( level: "normal", shares: 50 )
         ),
         unitNumber: 7,
         uptCompatibilityEnabled: false,
         wakeOnLanEnabled: true
       ),
       operation: "edit"
     )]
  ),
  customization: CustomizationSpec(
    encryptionKey: [48,
     *
     *     # Too much numbers were here, removing too make the thread look smaller
     *
     *
     30],
    globalIPSettings: CustomizationGlobalIPSettings(
      dnsServerList: [],
      dnsSuffixList: []
    ),
    identity: CustomizationSysprep(
      guiRunOnce: CustomizationGuiRunOnce(
        commandList: ["net stop myservice",
         "reg delete my_reg_key"]
      ),
      guiUnattended: CustomizationGuiUnattended(
        autoLogon: true,
        autoLogonCount: 1,
        password: CustomizationPassword(
          plainText: false,
          value: "XXX"
        ),
        timeZone: 33
      ),
      identification: CustomizationIdentification( joinWorkgroup: "WORKGROUP"
      ),
      userData: CustomizationUserData(
        computerName: CustomizationVirtualMachineName(  ),
        fullName: "IT Department",
        orgName: "My Organization",
        productId: ""
      )
    ),
    nicSettingMap: [CustomizationAdapterMapping(
       adapter: CustomizationIPSettings(
         dnsServerList: ["10.*.*.1", "10.*.*.2"],
         gateway: ["10.*.*.3", "10.*.*.4"],
         ip: CustomizationFixedIp( ipAddress: "10.*.*.5" ),
         primaryWINS: "",
         secondaryWINS: "",
         subnetMask: "255.*.*.0"
       )
     )],
    options: CustomizationWinOptions( changeSID: true, deleteAccounts: false )
  ),
  location: VirtualMachineRelocateSpec(
    datastore: Datastore("datastore-121"),
    pool: ResourcePool("resgroup-8")
  ),
  powerOn: false,
  template: false
)
Finished creating virtual machine testVM
Powered on virtual machine testVM

Expected Result:

Vm has to build with custom templates and get added to the distributed vlan passed using --cvlan

Actual Result:

Vm builded but it creates its own VLAN instead of joining an existing one

swalberg commented 6 years ago

Since it's giving a VirtualEthernetCardBackingInfo I think we're hitting the block here. The code doesn't log anything and the rescue will catch anything, so this could be a variety of things.

Is the VLAN name correct? Is it on a distributed switch? Are there maybe multiple VLANs with that name?

If you can find vsphere_vm_clone.rb you could edit Line 662/663 from this:

   rescue
          # not connected to a distibuted switch?

to this:

  rescue Exception => e
     puts "Couldn't find backing info for the VLAN. I got #{e} instead"

that'll at least tell us why we're getting there.

Zorrom commented 6 years ago

Hi Sean, Let me see those changes and get back to you on this.

Zorrom commented 6 years ago

Hi Swal,

Yes the VLAN name is correct. I can see the exam vlan name as below when i the following command knife vsphere vlan list VLAN: VLAN01 # my vlan
Yes the vlan is on a ditributed switch which vlan01(dvSwitch3)
No there are no multiple vlans of the same vlan number

I made the changes to the vpshere_vm_clone.rb and I got the following error when tried to run the vm clone command

/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/knife-vsphere-2.0.1/lib/chef/knife/vsphere_vm_clone.rb:647:in `rescue in block in generate_clone_spec': uninitialized constant Chef::Knife::VsphereVmClone::Exception  (NameError)
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/knife-vsphere-2.0.1/lib/chef/knife/vsphere_vm_clone.rb:635:in `block in generate_clone_spec'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/knife-vsphere-2.0.1/lib/chef/knife/vsphere_vm_clone.rb:633:in `each'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/knife-vsphere-2.0.1/lib/chef/knife/vsphere_vm_clone.rb:633:in `each_with_index'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/knife-vsphere-2.0.1/lib/chef/knife/vsphere_vm_clone.rb:633:in `generate_clone_spec'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/knife-vsphere-2.0.1/lib/chef/knife/vsphere_vm_clone.rb:368:in `run'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/knife.rb:443:in `block in run_with_pretty_exceptions'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/local_mode.rb:44:in `with_server_connectivity'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/knife.rb:442:in `run_with_pretty_exceptions'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/knife.rb:219:in `run'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/knife.rb:156:in `run'
        from /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/bin/knife:25:in `<top (required)>'
        from /bin/knife:57:in `load'
        from /bin/knife:57:in `<main>'

swalberg commented 6 years ago

That's weird. Maybe try rescue => e for that first line? I can try a couple of things when I get into work if that doesn't work out.

Zorrom commented 6 years ago

Our Infra is currently under a bit of maintenance kind of thing. So I couldnt check anything as of now. Will get back on this a bit later

SandhyaBadgujar commented 6 years ago

Hi I am trying the similar cmd for VM Clone and instead of changing the vlan a new vlan is getting created under dvSwitch3. Is the above issue resolved? looking for solution to this problem.

swalberg commented 6 years ago

Hi, not that I know of. I wasn't able to reproduce the problem. If you want to try adding that debugging output mentioned above, it might help me figure out why it's doing that.

Zorrom commented 6 years ago

Hi Sea, Sorry we got stuck with different objectives and so we have to put this on hold. So as per the error we found that our vCetner is designed in a way that everytime a Vlan is changed it would require MAC address to be passed with it which could either be manual or automatic. So specifying the Vlan without mac address is creating a completely different network group in the vCenter In the above command I passed the MAC address using --cmacs and specified my custom mac address along with vlan and this time the VM got allocated to the existing vLan that we have specified. However we got an error message during VM power on as follows

RbVmomi::Fault: GenericVmConfigFault: Invalid VPX-assigned Ethernet address: 'my_mac_address_here'.

To avoid this error we splitted the VM build operation and it was easy because we have an orchestrator doing all these steps.

Build VM without vlan and take the default vlan from the template
Delete the existing vlan
Add the vlan with the custom mac address
Fetch Ip of the machine and bootstrap with Chef - And these steps work just fine

So the error during power on is something coming from Chef or it's the vCenter issue.

swalberg commented 6 years ago

Thanks for the update. What you're seeing is coming from vsphere. I have a feeling that --cmacs option gets used for a couple of different things and you're in a corner case we didn't expect.

Crazy idea here, does it work if you pick a MAC address outside the vmware OID? The only other occurrence of that error message I see in Google comes from a python project where that was one solution. This may be more for curiosity as I don't recommend using a made up MAC, but if it works, maybe I can figure out their eventual solution.

Zorrom commented 6 years ago

Not exactly. Once the machine is built , I am deleting the old network adapters and replacing them with a new network adapter. It was at this point I specify my VLAN along with my Mac address. This step works fine when the VLAN operation is carried out separately and only becomes a problem when passing both Mac and VLAN using knife vsphere command.

chef / knife-vsphere