vmware / vic

vSphere Integrated Containers Engine is a container runtime for vSphere.
http://vmware.github.io/vic
Other
639 stars 173 forks source link

Longevity Failure: Error response from daemon: Server error from portlayer: A specified parameter was not correct: path #3022

Closed mhagen-vmware closed 7 years ago

mhagen-vmware commented 7 years ago

14-1-Longevity-VCH-0-4365-container-logs.zip

mhagen-vmware commented 7 years ago

Failing container: 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a

mhagen-vmware commented 7 years ago

From docker-personality.log:

time=2016-11-04T10:21:53.490248068Z level=debug msg=[ END ] [github.com/vmware/vic/lib/apiservers/engine/backends.(*Container).ContainerCreate:219] [2.572419941s]  
time="2016-11-04T10:21:53Z" level=debug msg="Calling POST /v1.23/containers/9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a/start" 
time="2016-11-04T10:21:53Z" level=debug msg="Client and server don't have the same version (client: 1.11.2, server: 1.22)" 
time=2016-11-04T10:21:53.513064975Z level=debug msg=[BEGIN] [github.com/vmware/vic/lib/apiservers/engine/backends.(*Container).ContainerStart:482] 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a 
time="2016-11-04T10:21:53Z" level=debug msg="mapPorts for \"9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a\": map[]" 
time="2016-11-04T10:22:37Z" level=debug msg="UnmapPorts: map[]" 
time=2016-11-04T10:22:37.793960639Z level=debug msg=[ END ] [github.com/vmware/vic/lib/apiservers/engine/backends.(*Container).ContainerStart:482] [44.28089206s] 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a 
time="2016-11-04T10:22:37Z" level=error msg="Handler for POST /v1.23/containers/9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a/start returned error: Server error from portlayer: A specified parameter was not correct: path" 
mhagen-vmware commented 7 years ago

From portlayer.log:

time=2016-11-04T10:21:53.521698557Z level=debug msg=[BEGIN] [github.com/vmware/vic/lib/portlayer/exec.Commit:35] 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a 
time=2016-11-04T10:21:53.521828714Z level=debug msg=[BEGIN] [github.com/vmware/vic/lib/portlayer/exec.(*containerBase).updates:98] 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a 
time=2016-11-04T10:21:53.527299804Z level=debug msg=[ END ] [github.com/vmware/vic/lib/portlayer/exec.(*containerBase).updates:98] [5.471232ms] 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a 
time=2016-11-04T10:21:55.314495706Z level=debug msg=[BEGIN] [github.com/vmware/vic/lib/portlayer/exec.(*Container).start:308] 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a 
time="2016-11-04T10:21:55Z" level=debug msg="Setting container 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a state: Starting" 
time="2016-11-04T10:22:37Z" level=debug msg="unexpected fault on task retry : &types.InvalidArgument{RuntimeFault:types.RuntimeFault{MethodFault:types.MethodFault{FaultCause:(*types.LocalizedMethodFault)(nil), FaultMessage:[]types.LocalizableMessage(nil)}}, InvalidProperty:\"path\"}" 
time="2016-11-04T10:22:37Z" level=debug msg="Do not fix non invalid state error" 
time="2016-11-04T10:22:37Z" level=debug msg="Setting container 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a state: Starting" 
time=2016-11-04T10:22:37.791637892Z level=debug msg=[ END ] [github.com/vmware/vic/lib/portlayer/exec.(*Container).start:308] [42.477141071s] 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a 
time=2016-11-04T10:22:37.791776846Z level=debug msg=[BEGIN] [github.com/vmware/vic/lib/portlayer/exec.(*Container).RefreshFromHandle:291] a88ce526714247dc505c44af8e242f43 
time="2016-11-04T10:22:37Z" level=debug msg="container refreshed - ChangeVersion: 2016-11-04T10:22:40.751987Z" 
time=2016-11-04T10:22:37.792035126Z level=debug msg=[ END ] [github.com/vmware/vic/lib/portlayer/exec.(*Container).RefreshFromHandle:291] [256.857µs] a88ce526714247dc505c44af8e242f43 
time=2016-11-04T10:22:37.792162958Z level=debug msg=[ END ] [github.com/vmware/vic/lib/portlayer/exec.Commit:35] [44.270456071s] 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a 
time="2016-11-04T10:22:37Z" level=error msg="CommitHandler error on handle(a88ce526714247dc505c44af8e242f43) for 9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a: task.Error{LocalizedMethodFault:(*types.LocalizedMethodFault)(0xc4200ceb20)}" 
mlh78750 commented 7 years ago

hostd logs on homer don't seem to end with anything wrong for that container.

hickeng commented 7 years ago

From the hostd logs the VM is unregistered and registered - this suggests there was an Invalid VM state at some point:

2016-11-04T10:23:23.698Z info hostd[FFC80B70] [Originator@6876 sub=Libs] VsanComplianceEventCb: Object Entry Deleted. state 0 for 6e611c58-5f32-22de-e398-1866da12461f
2016-11-04T10:23:23.698Z info hostd[FFC80B70] [Originator@6876 sub=VmObjectStorageMonitor] VmObjectStateCallback(UUID: '6e611c58-5f32-22de-e398-1866da12461f', compliant: false, accessible: false, deleted: true)
2016-11-04T10:23:23.698Z warning hostd[FFC80B70] [Originator@6876 sub=Libs] Failed to process CMMDS update u:6e611c58-5f32-22de-e398-1866da12461f:t:13:o:00000000-0000-0000-0000-000000000000(Hlth:Unhealthy:0):r:2(FFC7FEAC)
2016-11-04T10:23:23.967Z info hostd[92780B70] [Originator@6876 sub=Vimsvc.TaskManager opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Task Created : haTask-249-vim.VirtualMachine.unregister-86486
2016-11-04T10:23:23.967Z info hostd[92780B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] State Transition (VM_STATE_OFF -> VM_STATE_UNREGISTERING)
2016-11-04T10:23:23.968Z info hostd[92780B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Unregistered monitor for 6e611c58-5f32-22de-e398-1866da12461f refcount 0  type 13: Success
2016-11-04T10:23:23.968Z info hostd[92780B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Unregistered monitor for 70611c58-1079-ad29-676c-1866da12461f refcount 0  type 13: Success
2016-11-04T10:23:23.968Z warning hostd[92780B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] No VM found in reload state map for MOID '70611c58-1079-ad29-676c-1866da12461f', (object '249')
2016-11-04T10:23:23.968Z info hostd[92780B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Unregistered monitor for c75f1c58-eb07-1e18-2cc5-1866da12456b refcount 0  type 13: Success
2016-11-04T10:23:23.968Z warning hostd[92780B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] No VM found in reload state map for MOID 'c75f1c58-eb07-1e18-2cc5-1866da12456b', (object '249')
2016-11-04T10:23:23.968Z info hostd[92780B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Unregistered monitor for d05f1c58-c765-03d1-afaa-1866da12456b refcount 0  type 13: Success
2016-11-04T10:23:23.968Z warning hostd[92780B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] No VM found in reload state map for MOID 'd05f1c58-c765-03d1-afaa-1866da12456b', (object '249')
2016-11-04T10:23:23.968Z info hostd[92780B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Unregistered monitor for d85f1c58-2bf6-cf24-14f7-1866da12456b refcount 0  type 13: Success
2016-11-04T10:23:23.968Z warning hostd[92780B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] No VM found in reload state map for MOID 'd85f1c58-2bf6-cf24-14f7-1866da12456b', (object '249')
2016-11-04T10:23:23.969Z info hostd[91040B70] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 2528 : The dvPort 592 was disconnected in the vSphere Distributed Switch  in ha-datacenter.
2016-11-04T10:23:23.969Z info hostd[91040B70] [Originator@6876 sub=Guestsvc.GuestFileTransferImpl] VmOperationListener: unregister notification received for VM: 249
2016-11-04T10:23:23.969Z info hostd[91040B70] [Originator@6876 sub=Guestsvc.GuestFileTransferImpl] VmOperationListener succeeded
2016-11-04T10:23:23.969Z info hostd[91040B70] [Originator@6876 sub=Hbrsvc] Replicator: UnregisterListener triggered for config VM 249
2016-11-04T10:23:23.969Z info hostd[92780B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Event 2529 : Removed condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a on homer.eng.vmware.com from ha-datacenter
2016-11-04T10:23:23.969Z info hostd[92780B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] State Transition (VM_STATE_UNREGISTERING -> VM_STATE_GONE)
2016-11-04T10:23:23.969Z info hostd[92780B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Virtual machine object cleanup
2016-11-04T10:23:23.969Z info hostd[92780B70] [Originator@6876 sub=Vimsvc.TaskManager opID=7b141efd-D9-45-7e-3b-48e2 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Task Completed : haTask-249-vim.VirtualMachine.unregister-86486 Status success
2016-11-04T10:23:24.007Z info hostd[927C1B70] [Originator@6876 sub=Hostsvc.DvsTracker opID=7b141efd-D9-45-7e-fc-de-48f4 user=vpxuser] FetchUplinkDVPortgroups: Appended uplinkPg for dvs bf ad 20 50 60 53 07 60-3c 55 24 34 82 4f f1 45 portgroup dvportgroup-22
2016-11-04T10:23:24.007Z info hostd[927C1B70] [Originator@6876 sub=Hostsvc.DvsTracker opID=7b141efd-D9-45-7e-fc-de-48f4 user=vpxuser] FetchUplinkDVPortgroups: added 3 items
2016-11-04T10:23:24.914Z info hostd[910F2B70] [Originator@6876 sub=Vimsvc.TaskManager opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Task Created : haTask-ha-folder-vm-vim.Folder.registerVm-86495
2016-11-04T10:23:24.915Z info hostd[92780B70] [Originator@6876 sub=Solo.HaVMFolder opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Register called: []/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx
2016-11-04T10:23:24.926Z info hostd[927C1B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] SetVmxVersion: switching environment browser to vmx version 'vmx-11'
2016-11-04T10:23:24.962Z warning hostd[927C1B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] CannotRetrieveCorefiles: VM is in an invalid state
2016-11-04T10:23:25.074Z info hostd[927C1B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Checking for all objects accessibility (VM's current state: VM_STATE_INITIALIZING, stable? false)
2016-11-04T10:23:25.074Z error hostd[927C1B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Object not being tracked by VM object monitor (UUID: '6e611c58-5f32-22de-e398-1866da12461f'), assumed Healthy
2016-11-04T10:23:25.074Z error hostd[927C1B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Object not being tracked by VM object monitor (UUID: '70611c58-1079-ad29-676c-1866da12461f'), assumed Healthy
2016-11-04T10:23:25.074Z error hostd[927C1B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Object not being tracked by VM object monitor (UUID: 'c75f1c58-eb07-1e18-2cc5-1866da12456b'), assumed Healthy
2016-11-04T10:23:25.074Z error hostd[927C1B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Object not being tracked by VM object monitor (UUID: 'd05f1c58-c765-03d1-afaa-1866da12456b'), assumed Healthy
2016-11-04T10:23:25.074Z error hostd[927C1B70] [Originator@6876 sub=VmObjectStorageMonitor opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Object not being tracked by VM object monitor (UUID: 'd85f1c58-2bf6-cf24-14f7-1866da12456b'), assumed Healthy
2016-11-04T10:23:25.074Z info hostd[92780B70] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 2530 : The dvPort 592 was connected in the vSphere Distributed Switch  in ha-datacenter
2016-11-04T10:23:25.074Z info hostd[92780B70] [Originator@6876 sub=Hbrsvc] Replicator: ReconfigListener failed to look up VM (id=250)
2016-11-04T10:23:25.114Z info hostd[927C1B70] [Originator@6876 sub=Hbrsvc opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Replicator: VmFileProviderCallback failed to look up VM (id=250)
2016-11-04T10:23:25.123Z info hostd[927C1B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Error getting extended config name from config file.
2016-11-04T10:23:25.123Z info hostd[927C1B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Error getting extended config file path.
2016-11-04T10:23:25.123Z info hostd[927C1B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Initial tools version: 7:guestToolsNotInstalled
2016-11-04T10:23:25.123Z info hostd[927C1B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] State Transition (VM_STATE_INITIALIZING -> VM_STATE_OFF)
2016-11-04T10:23:25.123Z info hostd[927C1B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Initialized virtual machine.
2016-11-04T10:23:25.136Z info hostd[927C1B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Registered monitor for 6e611c58-5f32-22de-e398-1866da12461f type 13: Success
2016-11-04T10:23:25.142Z info hostd[927C1B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Registered monitor for 70611c58-1079-ad29-676c-1866da12461f type 13: Success
2016-11-04T10:23:25.142Z info hostd[927C1B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Registered monitor for c75f1c58-eb07-1e18-2cc5-1866da12456b type 13: Success
2016-11-04T10:23:25.145Z info hostd[927C1B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Registered monitor for d05f1c58-c765-03d1-afaa-1866da12456b type 13: Success
2016-11-04T10:23:25.145Z info hostd[927C1B70] [Originator@6876 sub=Libs opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Registered monitor for d85f1c58-2bf6-cf24-14f7-1866da12456b type 13: Success
2016-11-04T10:23:25.145Z info hostd[927C1B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Updated active set of monitored objects for VM '250': /vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/6e611c58-5f32-22de-e398-1866da12461f/condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a.vmx (5 objects)
2016-11-04T10:23:25.145Z info hostd[927C1B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Event 2531 : Registered condescending_s-9f3098502806660d1b711058df6ffe634e74e9a36f888c0205eecba2b718fd0a on homer.eng.vmware.com in ha-datacenter
2016-11-04T10:23:25.145Z info hostd[927C1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=7b141efd-D9-45-7e-ab-48f5 user=vpxuser:com.vmware.vic.42207f9e-c9d2-e972-f7dc-6ffd924e84dc] Task Completed : haTask-ha-folder-vm-vim.Folder.registerVm-86495 Status success

No evidence of this occurring in the port-layer log.

fdawg4l commented 7 years ago

@hickeng took a look at this and it looks like we don't have enough information in vpxd.log. We only get the last 500 lines which doesn't include the whole spec which is failing. This is the spec for the task which includes the data that is returning the invalid parameter error and we can't do much debugging.

We need to 1- log more vpxd lines 2- Include the task id in the portlayer of the vsphere tasks we're starting and from where they're being started from.

fdawg4l commented 7 years ago

2- Include the task id in the portlayer of the vsphere tasks we're starting and from where they're being started from.

I'm working on a change to implement this. Hopefully it will give us more insight if this happens again.

@jzt are you looking at (1)?

jzt commented 7 years ago

@fdawg4l yep!

fdawg4l commented 7 years ago

Logging of tasks PR is #3222.

fdawg4l commented 7 years ago

Ok, so we'll at least see the vsphere task MOID and can correlate to the hostd/vpxd log from the port layer log. If it happens again, we'll hopefully be able to correlate the task to the error we're getting.

jzt commented 7 years ago

Currently roadblocked trying to figure out why vpxd.log gets truncated at 500 lines. @hickeng chime in if you have any ideas.

hickeng commented 7 years ago

Seen on ESX here: https://ci.vmware.run/vmware/vic/7144

fdawg4l commented 7 years ago

For ^^^

PL

107 time=2016-11-21T16:12:36.926064118Z level=debug msg=[BEGIN] [github.com/vmware/vic/pkg/vsphere/disk.(*Manager).CreateAndAttach:85] [datastore1] VCH-7144-6364/VIC/564dd448-4352-7e99-788b-8eb0cada27f0/images/scratch/scratch.vmdk·
108 time=2016-11-21T16:12:36.926079070Z level=info msg=op=253.4 (delta:561.892719ms): Create/attach vmdk [datastore1] VCH-7144-6364/VIC/564dd448-4352-7e99-788b-8eb0cada27f0/images/scratch/scratch.vmdk from parent··
109 time=2016-11-21T16:12:36.930286914Z level=debug msg=op=253.4 (delta:566.097473ms): Attach reconfigure task=Task:haTask-13120-vim.VirtualMachine.reconfigure-4801534·
110 time="2016-11-21T16:12:36Z" level=debug msg="unexpected fault on task retry : &types.InvalidState{VimFault:types.VimFault{MethodFault:types.MethodFault{FaultCause:(*types.LocalizedMethodFault)(nil), FaultMessage:[]types.LocalizableMessage(nil)}}}"·
111 time="2016-11-21T16:12:36Z" level=debug msg="Do not fix non invalid state error"·
112 time=2016-11-21T16:12:36.968518190Z level=error msg=op=253.4 (delta:604.330139ms): vmdk storage driver failed to attach disk: The operation is not allowed in the current state.·

hostd

 928 2016-11-21T16:12:36.348Z info hostd[7B880B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/577bad39-46bacc3a-720b-0050569c9c22/VCH-7144-6364/VCH-7144-6364.vmx opID=ee59e803 user=root] State Transition (VM_STATE_ON -> VM_STATE_GUEST_OPERATION)

...

 948 2016-11-21T16:12:37.435Z info hostd[77281B70] [Originator@6876 sub=Vimsvc.TaskManager opID=ee59e819 user=root] Task Created : haTask-13120-vim.VirtualMachine.reconfigure-4801534
 949 2016-11-21T16:12:37.436Z error hostd[77281B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/577bad39-46bacc3a-720b-0050569c9c22/VCH-7144-6364/VCH-7144-6364.vmx opID=ee59e819 user=root] Invalid transition requested (VM_STATE_GUEST_OPERATION -> VM_STATE_RECONFIGURING): Invalid state
 950 2016-11-21T16:12:37.436Z warning hostd[77281B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/577bad39-46bacc3a-720b-0050569c9c22/VCH-7144-6364/VCH-7144-6364.vmx opID=ee59e819 user=root] Failed to start Reconfiguring: vim.fault.InvalidState
 951 2016-11-21T16:12:37.436Z info hostd[77281B70] [Originator@6876 sub=Default opID=ee59e819 user=root] AdapterServer caught exception: vim.fault.InvalidState
 952 2016-11-21T16:12:37.436Z info hostd[77281B70] [Originator@6876 sub=Vimsvc.TaskManager opID=ee59e819 user=root] Task Completed : haTask-13120-vim.VirtualMachine.reconfigure-4801534 Status error
fdawg4l commented 7 years ago

@hickeng VM_STATE_GUEST_OPERATION -> VM_STATE_RECONFIGURING looks wrong.

fdawg4l commented 7 years ago

opID=ee59e803, which triggers the Guest Operation isn't mentioned anywhere else in the log. Infact the last time the vmx is mentioned was a config update and it finishes successfully.

fdawg4l commented 7 years ago

So the time slice we're interested in (where the VM goes from ON to Guest Operation) is the time before T16:12:36.348Z to T16:12:37.435Z in the hostd log where the end of our slice is where our reconfigure operation is started (which fails). The PL starts the reconfigure (according the the pl log) at T16:12:36.930286914Z.

fdawg4l commented 7 years ago

I'm wondering if decoding the vch config from guest info is the trigger. I'll try making this call on a VCH and checking what task / state gets created on hostd.

43 time="2016-11-21T16:12:35Z" level=debug msg="Decoded VCH config for network: network.Configuration{source:(extraconfig.DataSource)(0x6a2c90), sink:(extraconfig.DataSink)(0x6a2940)...

fdawg4l commented 7 years ago

For this latest failure, we're moving the discussion to #3273.

mdubya66 commented 7 years ago

Removing from the GA release. We now have the ability to get the logs if it happens again but this one is very rare and not reproducible.

hickeng commented 7 years ago

This happened in the longevity tests, and vpxd has it all. Uploading bundle, but the operation that's failing is below:

failing operations

2016-11-24T09:32:59.951Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VpxProfiler opID=274079b9-C2-c2-25] DVS@PortSender::SendToHost [DVS@PortSender::SendToHost] took 24753 ms
2016-11-24T09:32:59.956Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VpxProfiler opID=274079b9-C2-c2-25] DVS@VpxdInvtDVS::PreMigrateCallback [DVS@VpxdInvtDVS::PreMigrateCallback] took 24758 ms
2016-11-24T09:32:59.956Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] InvokeListenerPreCallback [DVS] took 24758 ms
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on EventEmitter
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on FT
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on InvtId
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on Iofilter
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on MetadataHandler
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on PbmCallBack
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: PbmFunctionTracer: Entering PreMigrateCallback
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: PbmFunctionTracer: Entering FillSpec
2016-11-24T09:32:59.956Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: FillConfigSpec:  ConfigSpec is NULL
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: ~PbmFunctionTracer: Leaving  FillSpec
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: ~PbmFunctionTracer: Leaving  PreMigrateCallback
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on PermissionsHandler
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on SDRS
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on VApp
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on DRM
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on ResourceReservation
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on OCM
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on StorageListeners
2016-11-24T09:32:59.972Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowImpl] Execute local action vpx.vmprov.UnregisterSourceVm (1%)
2016-11-24T09:33:00.045Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowImpl] Execute local action vpx.vmprov.SwitchHostAndPool (1%)
2016-11-24T09:33:00.060Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VmprovUtil opID=274079b9-C2-c2-25] [PatchConfig] Not persisting the serial port network backings. If you have VMs with network backings for serial ports, please set config option: vpxd.SerialPort.PersistNetworkBacking. 
2016-11-24T09:33:00.068Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowImpl] Execute local action vpx.vmprov.RegisterDestinationVm (1%)
2016-11-24T09:33:00.149Z error vpxd[7FDC50FC6700] [Originator@6876 sub=VmprovUtil opID=274079b9-C2-c2-25] [VpxdVmprovUtil] Unable to register Vm ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmx
2016-11-24T09:33:00.176Z error vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowImpl] Get exception while executing action vpx.vmprov.RegisterDestinationVm: vmodl.fault.InvalidArgument

<snip the verbose config spec>

2016-11-24T09:33:00.199Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.RegisterDestinationVm
2016-11-24T09:33:00.218Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.RegisterDestinationVm with output:
--> false
2016-11-24T09:33:00.220Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.SwitchHostAndPool
2016-11-24T09:33:00.220Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VmprovUtil opID=274079b9-C2-c2-25] [PatchConfig] Not persisting the serial port network backings. If you have VMs with network backings for serial ports, please set config option: vpxd.SerialPort.PersistNetworkBacking. 
2016-11-24T09:33:00.225Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.SwitchHostAndPool with output:
--> false
2016-11-24T09:33:00.226Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.UnregisterSourceVm
2016-11-24T09:33:01.182Z warning vpxd[7FDC515D2700] [Originator@6876 sub=VmprovUtil opID=274079b9-C2-c2-25-45] [PatchConfig] Not persisting the serial port network backings. If you have VMs with network backings for serial ports, please set config option: vpxd.SerialPort.PersistNetworkBacking. 
2016-11-24T09:33:01.227Z error vpxd[7FDC515D2700] [Originator@6876 sub=corehostSync opID=274079b9-C2-c2-25-45] [Vds::Sync::UpdatePortConnecteeInt] The vm the port [590] belongs to is being migrated , we cannot change its proxyHost
2016-11-24T09:33:01.230Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.UnregisterSourceVm with output:
--> false
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.InvokeCallbacks
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on AutoStartManager with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on ContextValidator with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on DAS with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on DVS with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=coreanchor opID=274079b9-C2-c2-25] [MoDVSwitch::UpdatePortProxyHostInt] updating port [590] proxy host to [host-10]
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on EventEmitter with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on FT with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on InvtId with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on Iofilter with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on MetadataHandler with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on PbmCallBack with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: PbmFunctionTracer: Entering PostMigrateCallback
2016-11-24T09:33:01.273Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: ShouldSkipPostMigrateCallback: post migrate callback is skipped - VM migration failed
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: ~PbmFunctionTracer: Leaving  PostMigrateCallback
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on PermissionsHandler with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [PostMigrateCallback]Hit a fault so not applying permissions
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on SDRS with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on VApp with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on DRM with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on ResourceReservation with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on OCM with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on StorageListeners with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.InvokeCallbacks with output:
--> false
2016-11-24T09:33:01.275Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.ReserveDirectory
2016-11-24T09:33:01.275Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.ReserveDirectory with output:
--> false
2016-11-24T09:33:01.293Z error vpxd[7FDC50FC6700] [Originator@6876 sub=vpxLro opID=274079b9-C2-c2-25] [VpxLRO] Unexpected Exception: vmodl.fault.InvalidArgument
2016-11-24T09:33:01.293Z info vpxd[7FDC50FC6700] [Originator@6876 sub=vpxLro opID=274079b9-C2-c2-25] [VpxLRO] -- FINISH task-internal-1414746
2016-11-24T09:33:01.293Z info vpxd[7FDC50FC6700] [Originator@6876 sub=Default opID=274079b9-C2-c2-25] [VpxLRO] -- ERROR task-internal-1414746 --  -- VmprovWorkflow: vmodl.fault.InvalidArgument:
--> Result:
--> (vmodl.fault.InvalidArgument) {
-->    faultCause = (vmodl.MethodFault) null, 
-->    invalidProperty = "path", 
-->    msg = ""
--> }
--> Args:
--> 

The full snippet with the config spec:

2016-11-24T09:32:59.951Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VpxProfiler opID=274079b9-C2-c2-25] DVS@PortSender::SendToHost [DVS@PortSender::SendToHost] took 24753 ms
2016-11-24T09:32:59.956Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VpxProfiler opID=274079b9-C2-c2-25] DVS@VpxdInvtDVS::PreMigrateCallback [DVS@VpxdInvtDVS::PreMigrateCallback] took 24758 ms
2016-11-24T09:32:59.956Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] InvokeListenerPreCallback [DVS] took 24758 ms
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on EventEmitter
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on FT
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on InvtId
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on Iofilter
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on MetadataHandler
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on PbmCallBack
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: PbmFunctionTracer: Entering PreMigrateCallback
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: PbmFunctionTracer: Entering FillSpec
2016-11-24T09:32:59.956Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: FillConfigSpec:  ConfigSpec is NULL
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: ~PbmFunctionTracer: Leaving  FillSpec
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: ~PbmFunctionTracer: Leaving  PreMigrateCallback
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on PermissionsHandler
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on SDRS
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on VApp
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on DRM
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on ResourceReservation
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on OCM
2016-11-24T09:32:59.956Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke pre callback on StorageListeners
2016-11-24T09:32:59.972Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowImpl] Execute local action vpx.vmprov.UnregisterSourceVm (1%)
2016-11-24T09:33:00.045Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowImpl] Execute local action vpx.vmprov.SwitchHostAndPool (1%)
2016-11-24T09:33:00.060Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VmprovUtil opID=274079b9-C2-c2-25] [PatchConfig] Not persisting the serial port network backings. If you have VMs with network backings for serial ports, please set config option: vpxd.SerialPort.PersistNetworkBacking. 
2016-11-24T09:33:00.068Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowImpl] Execute local action vpx.vmprov.RegisterDestinationVm (1%)
2016-11-24T09:33:00.149Z error vpxd[7FDC50FC6700] [Originator@6876 sub=VmprovUtil opID=274079b9-C2-c2-25] [VpxdVmprovUtil] Unable to register Vm ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmx
2016-11-24T09:33:00.176Z error vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowImpl] Get exception while executing action vpx.vmprov.RegisterDestinationVm: vmodl.fault.InvalidArgument
--> 
--> [backtrace begin] product: VMware VirtualCenter, version: 6.0.0, build: build-3634794, tag: vpxd
--> backtrace[00] libvmacore.so[0x003C6234]: Vmacore::System::Stacktrace::CaptureWork(unsigned int)
--> backtrace[01] libvmacore.so[0x001F01C3]: Vmacore::System::SystemFactoryImpl::CreateQuickBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)
--> backtrace[02] libvmacore.so[0x0019A11D]: Vmacore::Throwable::Throwable(std::string const&)
--> backtrace[03] libvmomi.so[0x0027C3B9]
--> backtrace[04] libvmomi.so[0x00273581]: Vmomi::Fault::InvalidArgument::Throw()
--> backtrace[05] vpxd[0x00B6E01F]
--> backtrace[06] vpxd[0x00B6EB9A]
--> backtrace[07] vpxd[0x01C749BD]
--> backtrace[08] vpxd[0x01D261A7]
--> backtrace[09] vpxd[0x01CF8C2D]
--> backtrace[10] vpxd[0x01CED74F]
--> backtrace[11] vpxd[0x01CF14D3]
--> backtrace[12] vpxd[0x01CF20FB]
--> backtrace[13] vpxd[0x00B610DB]
--> backtrace[14] vpxd[0x00B5852B]
--> backtrace[15] vpxd[0x00B5DFDC]
--> backtrace[16] vpxd[0x00B58455]
--> backtrace[17] vpxd[0x00B584C1]
--> backtrace[18] vpxd[0x01C45008]
--> backtrace[19] vpxd[0x01837433]
--> backtrace[20] libvmodlTypes.so[0x0139EDCB]
--> backtrace[21] vpxd[0x00B85699]
--> backtrace[22] vpxd[0x00B610DB]
--> backtrace[23] vpxd[0x00B5852B]
--> backtrace[24] vpxd[0x00B5DFDC]
--> backtrace[25] vpxd[0x00B58455]
--> backtrace[26] vpxd[0x00B6B62B]
--> backtrace[27] libvmacore.so[0x002FA672]
--> backtrace[28] libvmacore.so[0x002FE0D2]
--> backtrace[29] libvmacore.so[0x002FE2F0]
--> backtrace[30] libvmacore.so[0x003056DE]
--> backtrace[31] libvmacore.so[0x002F9E0B]
--> backtrace[32] libvmacore.so[0x003D0102]
--> backtrace[33] libpthread.so.0[0x00007806]
--> backtrace[34] libc.so.6[0x000DE9BD]
--> [backtrace end]
--> 
2016-11-24T09:33:00.181Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] Workflow context:
--> (vpx.vmprov.MigrateContext) {
-->    cbData = (vmodl.KeyAnyValue) [
-->       (vmodl.KeyAnyValue) {
-->          key = "pbmPreCheckSkipped", 
-->          value = false
-->       }, 
-->       (vmodl.KeyAnyValue) {
-->          key = "defaultProfileData", 
-->          value = (vmodl.KeyAnyValue) [
-->             (vmodl.KeyAnyValue) {
-->                key = "0x7fdc48137e90", 
-->                value = true
-->             }
-->          ]
-->       }, 
-->       (vmodl.KeyAnyValue) {
-->          key = "profileData", 
-->          value = (vmodl.KeyAnyValue) [
-->             (vmodl.KeyAnyValue) {
-->                key = "2000", 
-->                value = (vim.vm.DefinedProfileSpec) {
-->                   profileId = "aa6d5a82-1c88-45da-85d3-3d74b91a5bad", 
-->                   profileData = (vim.vm.ProfileRawData) {
-->                      extensionKey = "com.vmware.vim.sps", 
-->                      objectData = "<ns1:storageProfile xmlns:ns1="http://profile.policy.data.vasa.vim.vmware.com/xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns1:StorageProfile"><ns1:constraints/><ns1:createdBy>Temporary user handle</ns1:createdBy><ns1:creationTime>2016-10-05T11:44:53.323+00:00</ns1:creationTime><ns1:description>Storage policy used as default for Virtual SAN datastores</ns1:description><ns1:generationId>0</ns1:generationId><ns1:lastUpdatedBy>Temporary user handle</ns1:lastUpdatedBy><ns1:lastUpdatedTime>2016-10-05T11:44:53.323+00:00</ns1:lastUpdatedTime><ns1:name>Virtual SAN Default Storage Policy</ns1:name><ns1:profileId>aa6d5a82-1c88-45da-85d3-3d74b91a5bad</ns1:profileId></ns1:storageProfile>"
-->                   }
-->                }
-->             }
-->          ]
-->       }, 
-->       (vmodl.KeyAnyValue) {
-->          key = "CopySpec.srcDsAccessibleByDst", 
-->          value = false
-->       }, 
-->       (vmodl.KeyAnyValue) {
-->          key = "copySpec", 
-->          value = (nfc.CopySpec) []
-->       }, 
-->       (vmodl.KeyAnyValue) {
-->          key = "pbmPreSkipped", 
-->          value = false
-->       }, 
-->       (vmodl.KeyAnyValue) {
-->          key = "sdrs.prov", 
-->          value = 418430261
-->       }
-->    ], 
-->    srcVmState = (vpx.vmprov.VmState) {
-->       vm = 'vim.VirtualMachine:dc112ee5-1570-4c87-8d33-60c3fd66f00c:vm-9291', 
-->       vmPathName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmx", 
-->       powerState = "poweredOff", 
-->       config = (vim.vm.ConfigInfo) {
-->          changeVersion = "2016-11-24T09:32:33.682353Z", 
-->          modified = "1970-01-01T00:00:00Z", 
-->          name = "insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38", 
-->          guestFullName = "Other 3.x or later Linux (64-bit)", 
-->          version = "vmx-11", 
-->          uuid = "e2222ec4-bba6-58bc-a246-a01ca53fff2d", 
-->          instanceUuid = "502015b6-4101-44a1-1494-59296228db82", 
-->          npivWorldWideNameType = <unset>, 
-->          npivDesiredNodeWwns = <unset>, 
-->          npivDesiredPortWwns = <unset>, 
-->          npivTemporaryDisabled = true, 
-->          npivOnNonRdmDisks = <unset>, 
-->          locationId = "", 
-->          template = false, 
-->          guestId = "other3xLinux64Guest", 
-->          alternateGuestName = "", 
-->          annotation = "", 
-->          files = (vim.vm.FileInfo) {
-->             vmPathName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmx", 
-->             snapshotDirectory = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/", 
-->             suspendDirectory = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/", 
-->             logDirectory = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/", 
-->             ftMetadataDirectory = <unset>
-->          }, 
-->          tools = (vim.vm.ToolsConfigInfo) {
-->             toolsVersion = 0, 
-->             afterPowerOn = true, 
-->             afterResume = true, 
-->             beforeGuestStandby = true, 
-->             beforeGuestShutdown = true, 
-->             beforeGuestReboot = <unset>, 
-->             toolsUpgradePolicy = "manual", 
-->             pendingCustomization = <unset>, 
-->             syncTimeWithHost = false, 
-->             lastInstallInfo = (vim.vm.ToolsConfigInfo.ToolsLastInstallInfo) {
-->                counter = 0, 
-->                fault = (vmodl.MethodFault) null
-->             }, 
-->             upgradeRebootPredict = <unset>
-->          }, 
-->          flags = (vim.vm.FlagInfo) {
-->             disableAcceleration = false, 
-->             enableLogging = true, 
-->             useToe = false, 
-->             runWithDebugInfo = false, 
-->             monitorType = "release", 
-->             htSharing = "any", 
-->             snapshotDisabled = false, 
-->             snapshotLocked = false, 
-->             diskUuidEnabled = true, 
-->             virtualMmuUsage = "automatic", 
-->             virtualExecUsage = "hvAuto", 
-->             snapshotPowerOffBehavior = "powerOff", 
-->             recordReplayEnabled = false, 
-->             faultToleranceType = "unset"
-->          }, 
-->          consolePreferences = (vim.vm.ConsolePreferences) null, 
-->          defaultPowerOps = (vim.vm.DefaultPowerOpInfo) {
-->             powerOffType = "soft", 
-->             suspendType = "hard", 
-->             resetType = "soft", 
-->             defaultPowerOffType = "soft", 
-->             defaultSuspendType = "hard", 
-->             defaultResetType = "soft", 
-->             standbyAction = "checkpoint"
-->          }, 
-->          hardware = (vim.vm.VirtualHardware) {
-->             numCPU = 2, 
-->             numCoresPerSocket = 1, 
-->             memoryMB = 2048, 
-->             virtualICH7MPresent = false, 
-->             virtualSMCPresent = false, 
-->             device = (vim.vm.device.VirtualDevice) [
-->                (vim.vm.device.VirtualIDEController) {
-->                   key = 200, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "IDE 0", 
-->                      summary = "IDE 0"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = <unset>, 
-->                   unitNumber = <unset>, 
-->                   busNumber = 0, 
-->                   device = (int) [
-->                      3000
-->                   ]
-->                }, 
-->                (vim.vm.device.VirtualIDEController) {
-->                   key = 201, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "IDE 1", 
-->                      summary = "IDE 1"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = <unset>, 
-->                   unitNumber = <unset>, 
-->                   busNumber = 1, 
-->                }, 
-->                (vim.vm.device.VirtualPS2Controller) {
-->                   key = 300, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "PS2 controller 0", 
-->                      summary = "PS2 controller 0"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = <unset>, 
-->                   unitNumber = <unset>, 
-->                   busNumber = 0, 
-->                   device = (int) [
-->                      600, 
-->                      700
-->                   ]
-->                }, 
-->                (vim.vm.device.VirtualPCIController) {
-->                   key = 100, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "PCI controller 0", 
-->                      summary = "PCI controller 0"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = <unset>, 
-->                   unitNumber = <unset>, 
-->                   busNumber = 0, 
-->                   device = (int) [
-->                      500, 
-->                      12000, 
-->                      1000, 
-->                      4000
-->                   ]
-->                }, 
-->                (vim.vm.device.VirtualSIOController) {
-->                   key = 400, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "SIO controller 0", 
-->                      summary = "SIO controller 0"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = <unset>, 
-->                   unitNumber = <unset>, 
-->                   busNumber = 0, 
-->                   device = (int) [
-->                      9000, 
-->                      9001, 
-->                      9002
-->                   ]
-->                }, 
-->                (vim.vm.device.VirtualKeyboard) {
-->                   key = 600, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "Keyboard ", 
-->                      summary = "Keyboard"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 300, 
-->                   unitNumber = 0
-->                }, 
-->                (vim.vm.device.VirtualPointingDevice) {
-->                   key = 700, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "Pointing device", 
-->                      summary = "Pointing device; Device"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualPointingDevice.DeviceBackingInfo) {
-->                      deviceName = "", 
-->                      useAutoDetect = false, 
-->                      hostPointingDevice = "autodetect"
-->                   }, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 300, 
-->                   unitNumber = 1
-->                }, 
-->                (vim.vm.device.VirtualVideoCard) {
-->                   key = 500, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "Video card ", 
-->                      summary = "Video card"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 100, 
-->                   unitNumber = 0, 
-->                   videoRamSizeInKB = 4096, 
-->                   numDisplays = 1, 
-->                   useAutoDetect = false, 
-->                   enable3DSupport = false, 
-->                   enableMPTSupport = <unset>, 
-->                   use3dRenderer = "automatic", 
-->                   graphicsMemorySizeInKB = 262144
-->                }, 
-->                (vim.vm.device.VirtualVMCIDevice) {
-->                   key = 12000, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "VMCI device", 
-->                      summary = "Device on the virtual machine PCI bus that provides support for the virtual machine communication interface"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 100, 
-->                   unitNumber = 17, 
-->                   id = -1, 
-->                   allowUnrestrictedCommunication = false, 
-->                   filterEnable = true, 
-->                   filterInfo = (vim.vm.device.VirtualVMCIDevice.FilterInfo) null
-->                }, 
-->                (vim.vm.device.ParaVirtualSCSIController) {
-->                   key = 1000, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "SCSI controller 0", 
-->                      summary = "VMware paravirtual SCSI"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDevice.BackingInfo) null, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 100, 
-->                   unitNumber = 3, 
-->                   busNumber = 0, 
-->                   device = (int) [
-->                      2000
-->                   ], 
-->                   hotAddRemove = true, 
-->                   sharedBus = "noSharing", 
-->                   scsiCtlrUnitNumber = 7
-->                }, 
-->                (vim.vm.device.VirtualCdrom) {
-->                   key = 3000, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "CD/DVD drive 1", 
-->                      summary = "ISO [vsanDatastore] a6b03658-0099-0050-9f06-1866da12435b/V0.8.0-RC2-7232-191D796-bootstrap.iso"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualCdrom.IsoBackingInfo) {
-->                      fileName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/a6b03658-0099-0050-9f06-1866da12435b/V0.8.0-RC2-7232-191D796-bootstrap.iso", 
-->                      datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                      backingObjectId = <unset>
-->                   }, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) {
-->                      startConnected = true, 
-->                      allowGuestControl = true, 
-->                      connected = false, 
-->                      status = "untried"
-->                   }, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 200, 
-->                   unitNumber = 0
-->                }, 
-->                (vim.vm.device.VirtualDisk) {
-->                   key = 2000, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "Hard disk 1", 
-->                      summary = "8,000,000 KB"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
-->                      fileName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmdk", 
-->                      datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                      backingObjectId = "b0b33658-2083-fa1c-ed8a-1866da12461f", 
-->                      diskMode = "persistent", 
-->                      split = false, 
-->                      writeThrough = false, 
-->                      thinProvisioned = true, 
-->                      eagerlyScrub = <unset>, 
-->                      uuid = "6000C294-c9b5-1b3e-961b-4daea2a3a3a8", 
-->                      contentId = "85daf194c0e42e2410a69ee91aae671e", 
-->                      changeId = <unset>, 
-->                      parent = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
-->                         fileName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/a6b03658-0099-0050-9f06-1866da12435b/VIC/4220e2f9-cbdc-79ca-93e1-4aaa511a5a5a/images/9967c5ad88de8c101809f7f22d4774b6791fe46ac3033d57abf7ebb1dd8e36ee/9967c5ad88de8c101809f7f22d4774b6791fe46ac3033d57abf7ebb1dd8e36ee.vmdk", 
-->                         datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                         backingObjectId = "7eb33658-7856-04cc-1b2d-1866da12435b", 
-->                         diskMode = "persistent", 
-->                         split = <unset>, 
-->                         writeThrough = <unset>, 
-->                         thinProvisioned = true, 
-->                         eagerlyScrub = <unset>, 
-->                         uuid = "6000C294-c9b5-1b3e-961b-4daea2a3a3a8", 
-->                         contentId = "85daf194c0e42e2410a69ee91aae671e", 
-->                         changeId = <unset>, 
-->                         parent = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
-->                            fileName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/a6b03658-0099-0050-9f06-1866da12435b/VIC/4220e2f9-cbdc-79ca-93e1-4aaa511a5a5a/images/a61cd723bcf2b0ccaaa3b8f779dfca17040bd459d9e615b82a7ea17993ec59f2/a61cd723bcf2b0ccaaa3b8f779dfca17040bd459d9e615b82a7ea17993ec59f2.vmdk", 
-->                            datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                            backingObjectId = "77b33658-8c14-7659-6adc-1866da12435b", 
-->                            diskMode = "persistent", 
-->                            split = <unset>, 
-->                            writeThrough = <unset>, 
-->                            thinProvisioned = true, 
-->                            eagerlyScrub = <unset>, 
-->                            uuid = "6000C294-c9b5-1b3e-961b-4daea2a3a3a8", 
-->                            contentId = "4583488655999d73b6139a9a9e2256e8", 
-->                            changeId = <unset>, 
-->                            parent = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
-->                               fileName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/a6b03658-0099-0050-9f06-1866da12435b/VIC/4220e2f9-cbdc-79ca-93e1-4aaa511a5a5a/images/scratch/scratch.vmdk", 
-->                               datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                               backingObjectId = "bab03658-70af-3c1f-14d0-1866da12435b", 
-->                               diskMode = "persistent", 
-->                               split = <unset>, 
-->                               writeThrough = <unset>, 
-->                               thinProvisioned = true, 
-->                               eagerlyScrub = <unset>, 
-->                               uuid = "6000C294-c9b5-1b3e-961b-4daea2a3a3a8", 
-->                               contentId = "3264611b9ed8b229bd62461a7cf94494", 
-->                               changeId = <unset>, 
-->                               parent = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) null, 
-->                               deltaDiskFormat = <unset>, 
-->                               digestEnabled = false, 
-->                               deltaGrainSize = <unset>, 
-->                               deltaDiskFormatVariant = <unset>, 
-->                               sharing = <unset>
-->                            }, 
-->                            deltaDiskFormat = "redoLogFormat", 
-->                            digestEnabled = false, 
-->                            deltaGrainSize = <unset>, 
-->                            deltaDiskFormatVariant = "vsanSparseVariant", 
-->                            sharing = <unset>
-->                         }, 
-->                         deltaDiskFormat = "redoLogFormat", 
-->                         digestEnabled = false, 
-->                         deltaGrainSize = <unset>, 
-->                         deltaDiskFormatVariant = "vsanSparseVariant", 
-->                         sharing = <unset>
-->                      }, 
-->                      deltaDiskFormat = "redoLogFormat", 
-->                      digestEnabled = false, 
-->                      deltaGrainSize = <unset>, 
-->                      deltaDiskFormatVariant = "vsanSparseVariant", 
-->                      sharing = "sharingNone"
-->                   }, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 1000, 
-->                   unitNumber = 0, 
-->                   capacityInKB = 8000000, 
-->                   capacityInBytes = 8192000000, 
-->                   shares = (vim.SharesInfo) {
-->                      shares = 1000, 
-->                      level = "normal"
-->                   }, 
-->                   storageIOAllocation = (vim.StorageResourceManager.IOAllocationInfo) {
-->                      limit = -1, 
-->                      shares = (vim.SharesInfo) {
-->                         shares = 1000, 
-->                         level = "normal"
-->                      }, 
-->                      reservation = 0
-->                   }, 
-->                   diskObjectId = "1790-2000", 
-->                   vFlashCacheConfigInfo = (vim.vm.device.VirtualDisk.VFlashCacheConfigInfo) null, 
-->                }, 
-->                (vim.vm.device.VirtualVmxnet3) {
-->                   key = 4000, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "Network adapter 1", 
-->                      summary = "DVSwitch: bf ad 20 50 60 53 07 60-3c 55 24 34 82 4f f1 45"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualEthernetCard.DistributedVirtualPortBackingInfo) {
-->                      port = (vim.dvs.PortConnection) {
-->                         switchUuid = "bf ad 20 50 60 53 07 60-3c 55 24 34 82 4f f1 45", 
-->                         portgroupKey = "dvportgroup-25", 
-->                         portKey = "590", 
-->                         connectionCookie = 777553186
-->                      }
-->                   }, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) {
-->                      startConnected = true, 
-->                      allowGuestControl = true, 
-->                      connected = false, 
-->                      status = "untried"
-->                   }, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.PciBusSlotInfo) {
-->                      pciSlotNumber = 192
-->                   }, 
-->                   controllerKey = 100, 
-->                   unitNumber = 7, 
-->                   addressType = "assigned", 
-->                   macAddress = "00:50:56:a0:7b:f2", 
-->                   wakeOnLanEnabled = true, 
-->                   resourceAllocation = (vim.vm.device.VirtualEthernetCard.ResourceAllocation) {
-->                      reservation = 0, 
-->                      share = (vim.SharesInfo) {
-->                         shares = 50, 
-->                         level = "normal"
-->                      }, 
-->                      limit = -1
-->                   }, 
-->                   externalId = <unset>, 
-->                   uptCompatibilityEnabled = true, 
-->                }, 
-->                (vim.vm.device.VirtualSerialPort) {
-->                   key = 9000, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "Serial port 1", 
-->                      summary = "Remote tcp://0.0.0.0:2377"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualSerialPort.URIBackingInfo) {
-->                      serviceURI = "tcp://0.0.0.0:2377", 
-->                      direction = "client", 
-->                      proxyURI = <unset>
-->                   }, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) {
-->                      startConnected = false, 
-->                      allowGuestControl = true, 
-->                      connected = false, 
-->                      status = "untried"
-->                   }, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 400, 
-->                   unitNumber = 0, 
-->                   yieldOnPoll = true
-->                }, 
-->                (vim.vm.device.VirtualSerialPort) {
-->                   key = 9001, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "Serial port 2", 
-->                      summary = "File [vsanDatastore] insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/tether.debug"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualSerialPort.FileBackingInfo) {
-->                      fileName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/tether.debug", 
-->                      datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                      backingObjectId = <unset>
-->                   }, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) {
-->                      startConnected = true, 
-->                      allowGuestControl = true, 
-->                      connected = false, 
-->                      status = "untried"
-->                   }, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 400, 
-->                   unitNumber = 1, 
-->                   yieldOnPoll = true
-->                }, 
-->                (vim.vm.device.VirtualSerialPort) {
-->                   key = 9002, 
-->                   deviceInfo = (vim.Description) {
-->                      label = "Serial port 3", 
-->                      summary = "File [vsanDatastore] insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/output.log"
-->                   }, 
-->                   backing = (vim.vm.device.VirtualSerialPort.FileBackingInfo) {
-->                      fileName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/output.log", 
-->                      datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                      backingObjectId = <unset>
-->                   }, 
-->                   connectable = (vim.vm.device.VirtualDevice.ConnectInfo) {
-->                      startConnected = true, 
-->                      allowGuestControl = true, 
-->                      connected = false, 
-->                      status = "untried"
-->                   }, 
-->                   slotInfo = (vim.vm.device.VirtualDevice.BusSlotInfo) null, 
-->                   controllerKey = 400, 
-->                   unitNumber = 2, 
-->                   yieldOnPoll = true
-->                }
-->             ]
-->          }, 
-->          cpuAllocation = (vim.ResourceAllocationInfo) {
-->             reservation = 0, 
-->             expandableReservation = false, 
-->             limit = -1, 
-->             shares = (vim.SharesInfo) {
-->                shares = 2000, 
-->                level = "normal"
-->             }, 
-->             overheadLimit = <unset>
-->          }, 
-->          memoryAllocation = (vim.ResourceAllocationInfo) {
-->             reservation = 0, 
-->             expandableReservation = false, 
-->             limit = -1, 
-->             shares = (vim.SharesInfo) {
-->                shares = 20480, 
-->                level = "normal"
-->             }, 
-->             overheadLimit = <unset>
-->          }, 
-->          latencySensitivity = (vim.LatencySensitivity) {
-->             level = "normal", 
-->             sensitivity = <unset>
-->          }, 
-->          memoryHotAddEnabled = false, 
-->          cpuHotAddEnabled = false, 
-->          cpuHotRemoveEnabled = false, 
-->          hotPlugMemoryLimit = <unset>, 
-->          hotPlugMemoryIncrementSize = <unset>, 
-->          cpuAffinity = (vim.vm.AffinityInfo) null, 
-->          memoryAffinity = (vim.vm.AffinityInfo) null, 
-->          networkShaper = (vim.vm.NetworkShaperInfo) null, 
-->          extraConfig = (vim.option.OptionValue) [
-->             (vim.option.OptionValue) {
-->                key = "nvram", 
-->                value = "insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.nvram"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge0.present", 
-->                value = "TRUE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "svga.present", 
-->                value = "TRUE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge4.present", 
-->                value = "TRUE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge4.virtualDev", 
-->                value = "pcieRootPort"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge4.functions", 
-->                value = "8"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge5.present", 
-->                value = "TRUE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge5.virtualDev", 
-->                value = "pcieRootPort"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge5.functions", 
-->                value = "8"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge6.present", 
-->                value = "TRUE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge6.virtualDev", 
-->                value = "pcieRootPort"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge6.functions", 
-->                value = "8"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge7.present", 
-->                value = "TRUE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge7.virtualDev", 
-->                value = "pcieRootPort"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "pciBridge7.functions", 
-->                value = "8"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "hpet0.present", 
-->                value = "true"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "ethernet0.pciSlotNumber", 
-->                value = "192"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "ethernet0.uptCompatibility", 
-->                value = "true"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "disk.EnableUUID", 
-->                value = "true"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "answer.msg.disk.duplicateUUID", 
-->                value = "Yes"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "answer.msg.serial.file.open", 
-->                value = "Append"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "sched.mem.lpage.maxSharedPages", 
-->                value = "256"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "vmotion.checkpointSVGAPrimarySize", 
-->                value = "4194304"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "serial0.hardwareFlowControl", 
-->                value = "TRUE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "memory.noHotAddOver4GB", 
-->                value = "FALSE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "memory.maxGrow", 
-->                value = "512"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "tools.remindInstall", 
-->                value = "FALSE"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.started", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.detail.starttime", 
-->                value = "1479979923"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/User", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/Common/name", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..diagnostics.resurrections", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/cmd/Dir", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/attach", 
-->                value = "true"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions", 
-->                value = "37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/Common/ExecutionEnvironment", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./version/BuildNumber", 
-->                value = "7232"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..createtime", 
-->                value = "1479979904"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/Common/id", 
-->                value = "192"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./version/Version", 
-->                value = "v0.8.0-rc2"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/default", 
-->                value = "true"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./diagnostics/debug", 
-->                value = "1"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.diagnostics.resurrections", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "networks|bridge/Common/notes", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/Common/name", 
-->                value = "bridge"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/restart", 
-->                value = "false"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..networks|bridge.network.type", 
-->                value = "bridge"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./version/BuildDate", 
-->                value = "2016/11/24@03:46:44"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./common/name", 
-->                value = "insane_elion"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/Group", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./layerid", 
-->                value = "9967c5ad88de8c101809f7f22d4774b6791fe46ac3033d57abf7ebb1dd8e36ee"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./common/id", 
-->                value = "37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/cmd/Args~", 
-->                value = "/bin/top"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/openstdin", 
-->                value = "false"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/Common/id", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks", 
-->                value = "bridge"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "networks|bridge/network/Common/notes", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./common/ExecutionEnvironment", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/cmd/Env", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/pools|0/last", 
-->                value = "rBD//w=="
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/pools", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./version/State", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/runblock", 
-->                value = "false"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/diagnostics/debug", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/common/ExecutionEnvironment", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/common/notes", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/cmd/Path", 
-->                value = "/bin/top"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/Common/ExecutionEnvironment", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.status", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/stopSignal", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/common/id", 
-->                value = "37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/common/name", 
-->                value = "insane_elion"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.detail.createtime", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/cmd/Args", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/tty", 
-->                value = "false"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/static", 
-->                value = "false"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice..sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.detail.stoptime", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./repo", 
-->                value = "busybox"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "common/notes", 
-->                value = "<nil>"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./sessions|37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38/cmd/Env~", 
-->                value = "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/pools|0/first", 
-->                value = "rBAAAA=="
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./key", 
-->                value = "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlCT3dJQkFBSkJBS3U1a21IMHFsbE8xTFR3L3JZWE5nemJBekdGUTBaYVYwY3NSV0ZxVllmRDBHSjUwTU5ZCjRxS0xNOEZhUUxkWkF0QWtoSUkzc2VsWlhPRmZIREVWZzBVQ0F3RUFBUUpCQUo0dVFmaVpOcEV0WENHUUhBLy8KTVhxR0Exc0tOZjRJR1gwbCtvTFJRQmZkYzk4WWVIM1FGeWg3STJJYVdocDdGTlRXY3Y5MUs1blJZaFFqN3JqVwpXOEVDSVFEQXJQRWpXN0g2alBONk9LSmNZTXlrUUw3MkNGWElhcndyRnhmMFpxZ05zd0loQU9RcDZiMUJXUSthCjEyTFZNZ241Z05aenpUTHJzM1NYcWVaaWdrOFhBbDhuQWlBckNiWm5ib0VIOVZvaGorcjhkWWJnbmJIS0pUeDgKSXhOYkVMS0NIamdscndJaEFNOHJxM3ZYd1RsbEhGczJQK2hFcjVLSnRYR2VDT1U5N053dTZDbE9PQzRUQWlBdQp5OEJ4d3ovVE4zZ0d1RUJKMzlpbjhrZFdUTTh6cWlKbU13dDc5bk4vNGc9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo="
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./version/GitCommit", 
-->                value = "191d796"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "vmware.tools.internalversion", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "vmware.tools.requiredversion", 
-->                value = "9541"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "migrate.hostLogState", 
-->                value = "none"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "migrate.migrationId", 
-->                value = "0"
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "migrate.hostLog", 
-->                value = ""
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/ip/IP", 
-->                value = "AAAAAAAAAAAAAP//rBAAAg=="
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/ip/Mask", 
-->                value = "//8AAA=="
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/gateway/IP", 
-->                value = "AAAAAAAAAAAAAP//rBAAAQ=="
-->             }, 
-->             (vim.option.OptionValue) {
-->                key = "guestinfo.vice./networks|bridge/network/gateway/Mask", 
-->                value = "//8AAA=="
-->             }
-->          ], 
-->          swapPlacement = "inherit", 
-->          swapDirectory = <unset>, 
-->          preserveSwapOnPowerOff = <unset>, 
-->          bootOptions = (vim.vm.BootOptions) {
-->             bootDelay = 0, 
-->             enterBIOSSetup = false, 
-->             bootRetryEnabled = false, 
-->             bootRetryDelay = 10000, 
-->             networkBootProtocol = "ipv4", 
-->          }, 
-->          ftInfo = (vim.vm.FaultToleranceConfigInfo) null, 
-->          repConfig = (vim.vm.ReplicationConfigSpec) null, 
-->          vAppConfig = (vim.vApp.VmConfigInfo) null, 
-->          vAssertsEnabled = false, 
-->          changeTrackingEnabled = false, 
-->          firmware = "bios", 
-->          maxMksConnections = 40, 
-->          guestAutoLockEnabled = false, 
-->          managedBy = (vim.ext.ManagedByInfo) null, 
-->          memoryReservationLockedToMax = false, 
-->          initialOverhead = (vim.vm.ConfigInfo.OverheadInfo) {
-->             initialMemoryReservation = 180899840, 
-->             initialSwapReservation = 171638784
-->          }, 
-->          nestedHVEnabled = false, 
-->          vPMCEnabled = false, 
-->          scheduledHardwareUpgradeInfo = (vim.vm.ScheduledHardwareUpgradeInfo) {
-->             upgradePolicy = "never", 
-->             versionKey = <unset>, 
-->             scheduledHardwareUpgradeStatus = "none", 
-->             fault = (vmodl.MethodFault) null
-->          }, 
-->          forkConfigInfo = (vim.vm.ForkConfigInfo) {
-->             parentEnabled = false, 
-->             childForkGroupId = <unset>, 
-->             childType = <unset>
-->          }, 
-->          vFlashCacheReservation = 0, 
-->          vmxConfigChecksum = 
-->             0202d465 ddce4015 121ea44a 1282069d
-->             1cf4a66c, 
-->          messageBusQueuePattern = <unset>, 
-->          messageBusTunnelEnabled = false, 
-->          vmStorageObjectId = <unset>, 
-->          swapStorageObjectId = <unset>, 
-->       }, 
-->       layoutEx = (vim.vm.FileLayoutEx) {
-->          file = (vim.vm.FileLayoutEx.FileInfo) [
-->             (vim.vm.FileLayoutEx.FileInfo) {
-->                key = 0, 
-->                name = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmx", 
-->                type = "config", 
-->                size = 7946, 
-->                uniqueSize = 7946, 
-->                backingObjectId = <unset>, 
-->                accessible = true
-->             }, 
-->             (vim.vm.FileLayoutEx.FileInfo) {
-->                key = 1, 
-->                name = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmsd", 
-->                type = "snapshotList", 
-->                size = 0, 
-->                uniqueSize = 0, 
-->                backingObjectId = <unset>, 
-->                accessible = true
-->             }, 
-->             (vim.vm.FileLayoutEx.FileInfo) {
-->                key = 2, 
-->                name = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/a6b03658-0099-0050-9f06-1866da12435b/VIC/4220e2f9-cbdc-79ca-93e1-4aaa511a5a5a/images/scratch/scratch.vmdk", 
-->                type = "diskDescriptor", 
-->                size = 394264576, 
-->                uniqueSize = 394264576, 
-->                backingObjectId = "bab03658-70af-3c1f-14d0-1866da12435b", 
-->                accessible = true
-->             }, 
-->             (vim.vm.FileLayoutEx.FileInfo) {
-->                key = 3, 
-->                name = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/a6b03658-0099-0050-9f06-1866da12435b/VIC/4220e2f9-cbdc-79ca-93e1-4aaa511a5a5a/images/a61cd723bcf2b0ccaaa3b8f779dfca17040bd459d9e615b82a7ea17993ec59f2/a61cd723bcf2b0ccaaa3b8f779dfca17040bd459d9e615b82a7ea17993ec59f2.vmdk", 
-->                type = "diskDescriptor", 
-->                size = 67108864, 
-->                uniqueSize = 67108864, 
-->                backingObjectId = "77b33658-8c14-7659-6adc-1866da12435b", 
-->                accessible = true
-->             }, 
-->             (vim.vm.FileLayoutEx.FileInfo) {
-->                key = 4, 
-->                name = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/a6b03658-0099-0050-9f06-1866da12435b/VIC/4220e2f9-cbdc-79ca-93e1-4aaa511a5a5a/images/9967c5ad88de8c101809f7f22d4774b6791fe46ac3033d57abf7ebb1dd8e36ee/9967c5ad88de8c101809f7f22d4774b6791fe46ac3033d57abf7ebb1dd8e36ee.vmdk", 
-->                type = "diskDescriptor", 
-->                size = 8388608, 
-->                uniqueSize = 8388608, 
-->                backingObjectId = "7eb33658-7856-04cc-1b2d-1866da12435b", 
-->                accessible = true
-->             }, 
-->             (vim.vm.FileLayoutEx.FileInfo) {
-->                key = 5, 
-->                name = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmdk", 
-->                type = "diskDescriptor", 
-->                size = 0, 
-->                uniqueSize = 0, 
-->                backingObjectId = "b0b33658-2083-fa1c-ed8a-1866da12461f", 
-->                accessible = true
-->             }
-->          ], 
-->          disk = (vim.vm.FileLayoutEx.DiskLayout) [
-->             (vim.vm.FileLayoutEx.DiskLayout) {
-->                key = 2000, 
-->                chain = (vim.vm.FileLayoutEx.DiskUnit) [
-->                   (vim.vm.FileLayoutEx.DiskUnit) {
-->                      fileKey = (int) [
-->                         2
-->                      ]
-->                   }, 
-->                   (vim.vm.FileLayoutEx.DiskUnit) {
-->                      fileKey = (int) [
-->                         3
-->                      ]
-->                   }, 
-->                   (vim.vm.FileLayoutEx.DiskUnit) {
-->                      fileKey = (int) [
-->                         4
-->                      ]
-->                   }, 
-->                   (vim.vm.FileLayoutEx.DiskUnit) {
-->                      fileKey = (int) [
-->                         5
-->                      ]
-->                   }
-->                ]
-->             }
-->          ], 
-->          timestamp = "2016-11-24T09:32:34.990347Z", 
-->       }, 
-->       capability = (vim.vm.Capability) {
-->          snapshotOperationsSupported = true, 
-->          multipleSnapshotsSupported = true, 
-->          snapshotConfigSupported = true, 
-->          poweredOffSnapshotsSupported = true, 
-->          memorySnapshotsSupported = true, 
-->          revertToSnapshotSupported = true, 
-->          quiescedSnapshotsSupported = true, 
-->          disableSnapshotsSupported = false, 
-->          lockSnapshotsSupported = true, 
-->          consolePreferencesSupported = false, 
-->          cpuFeatureMaskSupported = true, 
-->          s1AcpiManagementSupported = true, 
-->          settingScreenResolutionSupported = false, 
-->          toolsAutoUpdateSupported = false, 
-->          vmNpivWwnSupported = true, 
-->          npivWwnOnNonRdmVmSupported = true, 
-->          vmNpivWwnDisableSupported = true, 
-->          vmNpivWwnUpdateSupported = true, 
-->          swapPlacementSupported = true, 
-->          swapPreservationSupported = true, 
-->          toolsSyncTimeSupported = true, 
-->          virtualMmuUsageSupported = true, 
-->          diskSharesSupported = true, 
-->          bootOptionsSupported = true, 
-->          bootRetryOptionsSupported = true, 
-->          settingVideoRamSizeSupported = true, 
-->          settingDisplayTopologySupported = false, 
-->          settingDisplayTopologyModesSupported = false, 
-->          recordReplaySupported = true, 
-->          changeTrackingSupported = true, 
-->          multipleCoresPerSocketSupported = true, 
-->          hostBasedReplicationSupported = true, 
-->          guestAutoLockSupported = true, 
-->          memoryReservationLockSupported = true, 
-->          featureRequirementSupported = true, 
-->          poweredOnMonitorTypeChangeSupported = true, 
-->          vmfsNativeSnapshotSupported = false, 
-->          seSparseDiskSupported = true, 
-->          nestedHVSupported = true, 
-->          vPMCSupported = true, 
-->          toolsRebootPredictSupported = false, 
-->          messageBusSupported = false, 
-->          canConnectUSBDevices = <unset>
-->       }, 
-->    }, 
-->    srcLocation = (vpx.vmprov.VmLocation) {
-->       service = (vpx.vmprov.ServiceEndpointState) {
-->          instanceName = "bart.eng.vmware.com", 
-->          url = "https://bart.eng.vmware.com:443/sdk", 
-->          about = (vim.AboutInfo) {
-->             name = "VMware VirtualCenter", 
-->             fullName = "VMware VirtualCenter Server", 
-->             vendor = "", 
-->             version = "6.0.0", 
-->             build = "3634794", 
-->             localeVersion = <unset>, 
-->             localeBuild = <unset>, 
-->             osType = "linux-x64", 
-->             productLineId = "vpx", 
-->             apiType = "VirtualCenter", 
-->             apiVersion = "6.0", 
-->             instanceUuid = "dc112ee5-1570-4c87-8d33-60c3fd66f00c", 
-->             licenseProductName = "VMware VirtualCenter Server", 
-->             licenseProductVersion = "6.0"
-->          }
-->       }, 
-->       datacenter = 'vim.Datacenter:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datacenter-2', 
-->       folder = <unset>, 
-->       computeResource = 'vim.ClusterComputeResource:dc112ee5-1570-4c87-8d33-60c3fd66f00c:domain-c7', 
-->       pool = 'vim.VirtualApp:dc112ee5-1570-4c87-8d33-60c3fd66f00c:resgroup-v9268', 
-->       host = (vpx.vmprov.HostState) {
-->          host = 'vim.HostSystem:dc112ee5-1570-4c87-8d33-60c3fd66f00c:host-10', 
-->          product = (vim.AboutInfo) {
-->             name = "VMware ESXi", 
-->             fullName = "VMware ESXi 6.0.0 build-3620759", 
-->             vendor = "VMware, Inc.", 
-->             version = "6.0.0", 
-->             build = "3620759", 
-->             localeVersion = "INTL", 
-->             localeBuild = "000", 
-->             osType = "vmnix-x86", 
-->             productLineId = "embeddedEsx", 
-->             apiType = "HostAgent", 
-->             apiVersion = "6.0", 
-->             instanceUuid = <unset>, 
-->             licenseProductName = "VMware ESX Server", 
-->             licenseProductVersion = "6.0"
-->          }, 
-->          uuid = "4c4c4544-0030-4710-8051-b4c04f373732", 
-->          managementIp = "10.17.109.48", 
-->          cnxSpec = (nfc.CopySpec.CnxSpec) {
-->             host = "10.17.109.48", 
-->             port = 902, 
-->             authData = (nfc.CopySpec.UsernamePasswd) {
-->                username = "vpxuser", 
-->                password = (not shown), 
-->                sslThumbprint = "00:0E:CB:0D:FE:CE:69:AA:4C:1A:B3:01:A3:9A:9C:8B:BF:7C:EF:AB"
-->             }, 
-->             useSSL = true
-->          }, 
-->          sslThumbprint = "00:0E:CB:0D:FE:CE:69:AA:4C:1A:B3:01:A3:9A:9C:8B:BF:7C:EF:AB", 
-->          capability = (vim.host.Capability) {
-->             recursiveResourcePoolsSupported = true, 
-->             cpuMemoryResourceConfigurationSupported = true, 
-->             rebootSupported = true, 
-->             shutdownSupported = true, 
-->             vmotionSupported = true, 
-->             vmotionEncryptionSupported = false, 
-->             standbySupported = true, 
-->             ipmiSupported = true, 
-->             maxSupportedVMs = <unset>, 
-->             maxRunningVMs = 0, 
-->             maxSupportedVcpus = <unset>, 
-->             maxRegisteredVMs = 512, 
-->             datastorePrincipalSupported = false, 
-->             sanSupported = true, 
-->             nfsSupported = true, 
-->             iscsiSupported = true, 
-->             vlanTaggingSupported = true, 
-->             nicTeamingSupported = true, 
-->             highGuestMemSupported = true, 
-->             maintenanceModeSupported = true, 
-->             suspendedRelocateSupported = true, 
-->             restrictedSnapshotRelocateSupported = true, 
-->             perVmSwapFiles = true, 
-->             localSwapDatastoreSupported = true, 
-->             unsharedSwapVMotionSupported = true, 
-->             backgroundSnapshotsSupported = false, 
-->             preAssignedPCIUnitNumbersSupported = true, 
-->             screenshotSupported = true, 
-->             scaledScreenshotSupported = true, 
-->             storageVMotionSupported = true, 
-->             vmotionWithStorageVMotionSupported = true, 
-->             vmotionAcrossNetworkSupported = true, 
-->             maxNumDisksSVMotion = 188, 
-->             hbrNicSelectionSupported = true, 
-->             vrNfcNicSelectionSupported = true, 
-->             recordReplaySupported = true, 
-->             ftSupported = false, 
-->             replayUnsupportedReason = <unset>, 
-->             checkpointFtSupported = false, 
-->             smpFtSupported = false, 
-->             ftCompatibilityIssues = (string) [
-->                "haAgentIssue", 
-->                "missingFTLoggingNic", 
-->                "missingVMotionNic"
-->             ], 
-->             smpFtCompatibilityIssues = (string) [
-->                "haAgentIssue", 
-->                "missingFTLoggingNic", 
-->                "missingVMotionNic"
-->             ], 
-->             maxVcpusPerFtVm = 4, 
-->             loginBySSLThumbprintSupported = true, 
-->             cloneFromSnapshotSupported = true, 
-->             deltaDiskBackingsSupported = true, 
-->             perVMNetworkTrafficShapingSupported = false, 
-->             tpmSupported = false, 
-->             virtualExecUsageSupported = true, 
-->             storageIORMSupported = true, 
-->             vmDirectPathGen2Supported = true, 
-->             vmDirectPathGen2UnsupportedReasonExtended = <unset>, 
-->             supportedVmfsMajorVersion = (int) [
-->                5
-->             ], 
-->             vStorageCapable = true, 
-->             snapshotRelayoutSupported = true, 
-->             firewallIpRulesSupported = true, 
-->             servicePackageInfoSupported = true, 
-->             maxHostRunningVms = 128, 
-->             maxHostSupportedVcpus = 128, 
-->             advancedSystemResourcesSupported = <unset>, 
-->             vmfsDatastoreMountCapable = true, 
-->             eightPlusHostVmfsSharedAccessSupported = true, 
-->             nestedHVSupported = true, 
-->             vPMCSupported = true, 
-->             interVMCommunicationThroughVMCISupported = false, 
-->             scheduledHardwareUpgradeSupported = true, 
-->             featureCapabilitiesSupported = true, 
-->             latencySensitivitySupported = true, 
-->             storagePolicySupported = true, 
-->             accel3dSupported = false, 
-->             reliableMemoryAware = true, 
-->             multipleNetworkStackInstanceSupported = true, 
-->             messageBusProxySupported = true, 
-->             vsanSupported = true, 
-->             vFlashSupported = true, 
-->             hostAccessManagerSupported = true, 
-->             provisioningNicSelectionSupported = true, 
-->             nfs41Supported = true, 
-->             turnDiskLocatorLedSupported = true, 
-->             virtualVolumeDatastoreSupported = true, 
-->             markAsSsdSupported = true, 
-->             markAsLocalSupported = true, 
-->             smartCardAuthenticationSupported = true, 
-->          }, 
-->          internalCapability = (vim.host.InternalCapability) {
-->             enumerateExtentsSupported = true, 
-->             fastSuspendResumeSupported = true, 
-->             svmVmxDiskCopySupported = true, 
-->             evcSoftwareSupported = true, 
-->             lunDisplayNameSupported = true, 
-->             virtualDiskDigestSupported = true, 
-->             hostProfileEngineSupported = true, 
-->             controllableStatsCountersSupported = true, 
-->             nfcCopyRawDiskMappingSupported = true, 
-->             nfcCopyDeltaDiskSupported = true, 
-->             vmotionHostLogStateSupported = true, 
-->             operationCleanupManagerSupported = true, 
-->          }, 
-->          vmSwapPlacement = "vmDirectory", 
-->          localSwapDatastore = (vpx.vmprov.DatastoreState) null, 
-->          localSwapDatastoreWritable = <unset>, 
-->          configOption = (vim.vm.ConfigOption) null, 
-->          vNicInfo = (vim.host.VirtualNicManagerInfo) null, 
-->          network = (vim.host.NetworkInfo) null, 
-->          provPolicy = (vim.vm.ProvisioningPolicy) null, 
-->          vFlashModules = (vim.host.VFlashManager.VFlashCacheConfigInfo.VFlashModuleConfigOption) [
-->             (vim.host.VFlashManager.VFlashCacheConfigInfo.VFlashModuleConfigOption) {
-->                vFlashModule = "vfc", 
-->                vFlashModuleVersion = "1.0.0.0", 
-->                minSupportedModuleVersion = "1.0.0.0", 
-->                cacheConsistencyType = (vim.option.ChoiceOption) {
-->                   valueIsReadonly = true, 
-->                   choiceInfo = (vim.ElementDescription) [
-->                      (vim.ElementDescription) {
-->                         label = "strong", 
-->                         summary = "Cache data consistency is guaranteed after a crash", 
-->                         key = "strong"
-->                      }
-->                   ], 
-->                   defaultIndex = 0
-->                }, 
-->                cacheMode = (vim.option.ChoiceOption) {
-->                   valueIsReadonly = true, 
-->                   choiceInfo = (vim.ElementDescription) [
-->                      (vim.ElementDescription) {
-->                         label = "write_thru", 
-->                         summary = "Writes to the cache cause writes to the underlying storage", 
-->                         key = "write_thru"
-->                      }
-->                   ], 
-->                   defaultIndex = 0
-->                }, 
-->                blockSizeInKBOption = (vim.option.LongOption) {
-->                   valueIsReadonly = <unset>, 
-->                   min = 4, 
-->                   max = 1024, 
-->                   defaultValue = 8
-->                }, 
-->                reservationInMBOption = (vim.option.LongOption) {
-->                   valueIsReadonly = <unset>, 
-->                   min = 4, 
-->                   max = 409600, 
-->                   defaultValue = 4
-->                }, 
-->                maxDiskSizeInKB = 17179869184
-->             }
-->          ], 
-->          vFlashResourceAccessible = false, 
-->       }, 
-->       primaryDatastore = (vpx.vmprov.DatastoreState) {
-->          datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->          summary = (vim.Datastore.Summary) {
-->             datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->             name = "vsanDatastore", 
-->             url = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/", 
-->             capacity = 2970573864960, 
-->             freeSpace = 2656504381440, 
-->             uncommitted = 34617360384, 
-->             accessible = true, 
-->             multipleHostAccess = true, 
-->             type = "vsan", 
-->             maintenanceMode = "normal"
-->          }, 
-->          capability = (vim.Datastore.Capability) {
-->             directoryHierarchySupported = true, 
-->             rawDiskMappingsSupported = true, 
-->             perFileThinProvisioningSupported = true, 
-->             storageIORMSupported = false, 
-->             nativeSnapshotSupported = false, 
-->             nativeSnapshotCopySupported = false, 
-->             topLevelDirectoryCreateSupported = false, 
-->             seSparseSupported = false
-->          }
-->       }, 
-->       datastore = (vpx.vmprov.DatastoreState) [
-->          (vpx.vmprov.DatastoreState) {
-->             datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->             summary = (vim.Datastore.Summary) {
-->                datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                name = "vsanDatastore", 
-->                url = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/", 
-->                capacity = 2970573864960, 
-->                freeSpace = 2656504381440, 
-->                uncommitted = 34617360384, 
-->                accessible = true, 
-->                multipleHostAccess = true, 
-->                type = "vsan", 
-->                maintenanceMode = "normal"
-->             }, 
-->             capability = (vim.Datastore.Capability) {
-->                directoryHierarchySupported = true, 
-->                rawDiskMappingsSupported = true, 
-->                perFileThinProvisioningSupported = true, 
-->                storageIORMSupported = false, 
-->                nativeSnapshotSupported = false, 
-->                nativeSnapshotCopySupported = false, 
-->                topLevelDirectoryCreateSupported = false, 
-->                seSparseSupported = false
-->             }
-->          }
-->       ]
-->    }, 
-->    dstVmPathName = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/9eb33658-e0c0-cfb6-59bf-1866da12461f/insane_elion-37325ce3d771fd15b26cef250fe4c80614da6aa83fc8277a846b32c53b597f38.vmx", 
-->    dstVm = 'vim.VirtualMachine:dc112ee5-1570-4c87-8d33-60c3fd66f00c:vm-9291', 
-->    dstLocation = (vpx.vmprov.VmLocation) {
-->       service = (vpx.vmprov.ServiceEndpointState) {
-->          instanceName = "bart.eng.vmware.com", 
-->          url = "https://bart.eng.vmware.com:443/sdk", 
-->          about = (vim.AboutInfo) {
-->             name = "VMware VirtualCenter", 
-->             fullName = "VMware VirtualCenter Server", 
-->             vendor = "", 
-->             version = "6.0.0", 
-->             build = "3634794", 
-->             localeVersion = <unset>, 
-->             localeBuild = <unset>, 
-->             osType = "linux-x64", 
-->             productLineId = "vpx", 
-->             apiType = "VirtualCenter", 
-->             apiVersion = "6.0", 
-->             instanceUuid = "dc112ee5-1570-4c87-8d33-60c3fd66f00c", 
-->             licenseProductName = "VMware VirtualCenter Server", 
-->             licenseProductVersion = "6.0"
-->          }
-->       }, 
-->       datacenter = 'vim.Datacenter:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datacenter-2', 
-->       folder = <unset>, 
-->       computeResource = 'vim.ClusterComputeResource:dc112ee5-1570-4c87-8d33-60c3fd66f00c:domain-c7', 
-->       pool = 'vim.VirtualApp:dc112ee5-1570-4c87-8d33-60c3fd66f00c:resgroup-v9268', 
-->       host = (vpx.vmprov.HostState) {
-->          host = 'vim.HostSystem:dc112ee5-1570-4c87-8d33-60c3fd66f00c:host-15', 
-->          product = (vim.AboutInfo) {
-->             name = "VMware ESXi", 
-->             fullName = "VMware ESXi 6.0.0 build-3620759", 
-->             vendor = "VMware, Inc.", 
-->             version = "6.0.0", 
-->             build = "3620759", 
-->             localeVersion = "INTL", 
-->             localeBuild = "000", 
-->             osType = "vmnix-x86", 
-->             productLineId = "embeddedEsx", 
-->             apiType = "HostAgent", 
-->             apiVersion = "6.0", 
-->             instanceUuid = <unset>, 
-->             licenseProductName = "VMware ESX Server", 
-->             licenseProductVersion = "6.0"
-->          }, 
-->          uuid = "4c4c4544-0058-4610-8051-c2c04f373732", 
-->          managementIp = "10.17.109.212", 
-->          cnxSpec = (nfc.CopySpec.CnxSpec) {
-->             host = "10.17.109.212", 
-->             port = 902, 
-->             authData = (nfc.CopySpec.UsernamePasswd) {
-->                username = "vpxuser", 
-->                password = (not shown), 
-->                sslThumbprint = "5D:1C:3D:92:7A:27:C7:28:74:A4:77:4D:A4:DA:D5:3A:FE:BA:73:82"
-->             }, 
-->             useSSL = true
-->          }, 
-->          sslThumbprint = "5D:1C:3D:92:7A:27:C7:28:74:A4:77:4D:A4:DA:D5:3A:FE:BA:73:82", 
-->          capability = (vim.host.Capability) {
-->             recursiveResourcePoolsSupported = true, 
-->             cpuMemoryResourceConfigurationSupported = true, 
-->             rebootSupported = true, 
-->             shutdownSupported = true, 
-->             vmotionSupported = true, 
-->             vmotionEncryptionSupported = false, 
-->             standbySupported = true, 
-->             ipmiSupported = true, 
-->             maxSupportedVMs = <unset>, 
-->             maxRunningVMs = 0, 
-->             maxSupportedVcpus = <unset>, 
-->             maxRegisteredVMs = 512, 
-->             datastorePrincipalSupported = false, 
-->             sanSupported = true, 
-->             nfsSupported = true, 
-->             iscsiSupported = true, 
-->             vlanTaggingSupported = true, 
-->             nicTeamingSupported = true, 
-->             highGuestMemSupported = true, 
-->             maintenanceModeSupported = true, 
-->             suspendedRelocateSupported = true, 
-->             restrictedSnapshotRelocateSupported = true, 
-->             perVmSwapFiles = true, 
-->             localSwapDatastoreSupported = true, 
-->             unsharedSwapVMotionSupported = true, 
-->             backgroundSnapshotsSupported = false, 
-->             preAssignedPCIUnitNumbersSupported = true, 
-->             screenshotSupported = true, 
-->             scaledScreenshotSupported = true, 
-->             storageVMotionSupported = true, 
-->             vmotionWithStorageVMotionSupported = true, 
-->             vmotionAcrossNetworkSupported = true, 
-->             maxNumDisksSVMotion = 188, 
-->             hbrNicSelectionSupported = true, 
-->             vrNfcNicSelectionSupported = true, 
-->             recordReplaySupported = true, 
-->             ftSupported = false, 
-->             replayUnsupportedReason = <unset>, 
-->             checkpointFtSupported = false, 
-->             smpFtSupported = false, 
-->             ftCompatibilityIssues = (string) [
-->                "haAgentIssue", 
-->                "missingFTLoggingNic", 
-->                "missingVMotionNic"
-->             ], 
-->             smpFtCompatibilityIssues = (string) [
-->                "haAgentIssue", 
-->                "missingFTLoggingNic", 
-->                "missingVMotionNic"
-->             ], 
-->             maxVcpusPerFtVm = 4, 
-->             loginBySSLThumbprintSupported = true, 
-->             cloneFromSnapshotSupported = true, 
-->             deltaDiskBackingsSupported = true, 
-->             perVMNetworkTrafficShapingSupported = false, 
-->             tpmSupported = false, 
-->             virtualExecUsageSupported = true, 
-->             storageIORMSupported = true, 
-->             vmDirectPathGen2Supported = true, 
-->             vmDirectPathGen2UnsupportedReasonExtended = <unset>, 
-->             supportedVmfsMajorVersion = (int) [
-->                5
-->             ], 
-->             vStorageCapable = true, 
-->             snapshotRelayoutSupported = true, 
-->             firewallIpRulesSupported = true, 
-->             servicePackageInfoSupported = true, 
-->             maxHostRunningVms = 128, 
-->             maxHostSupportedVcpus = 128, 
-->             advancedSystemResourcesSupported = <unset>, 
-->             vmfsDatastoreMountCapable = true, 
-->             eightPlusHostVmfsSharedAccessSupported = true, 
-->             nestedHVSupported = true, 
-->             vPMCSupported = true, 
-->             interVMCommunicationThroughVMCISupported = false, 
-->             scheduledHardwareUpgradeSupported = true, 
-->             featureCapabilitiesSupported = true, 
-->             latencySensitivitySupported = true, 
-->             storagePolicySupported = true, 
-->             accel3dSupported = false, 
-->             reliableMemoryAware = true, 
-->             multipleNetworkStackInstanceSupported = true, 
-->             messageBusProxySupported = true, 
-->             vsanSupported = true, 
-->             vFlashSupported = true, 
-->             hostAccessManagerSupported = true, 
-->             provisioningNicSelectionSupported = true, 
-->             nfs41Supported = true, 
-->             turnDiskLocatorLedSupported = true, 
-->             virtualVolumeDatastoreSupported = true, 
-->             markAsSsdSupported = true, 
-->             markAsLocalSupported = true, 
-->             smartCardAuthenticationSupported = true, 
-->          }, 
-->          internalCapability = (vim.host.InternalCapability) {
-->             enumerateExtentsSupported = true, 
-->             fastSuspendResumeSupported = true, 
-->             svmVmxDiskCopySupported = true, 
-->             evcSoftwareSupported = true, 
-->             lunDisplayNameSupported = true, 
-->             virtualDiskDigestSupported = true, 
-->             hostProfileEngineSupported = true, 
-->             controllableStatsCountersSupported = true, 
-->             nfcCopyRawDiskMappingSupported = true, 
-->             nfcCopyDeltaDiskSupported = true, 
-->             vmotionHostLogStateSupported = true, 
-->             operationCleanupManagerSupported = true, 
-->          }, 
-->          vmSwapPlacement = "vmDirectory", 
-->          localSwapDatastore = (vpx.vmprov.DatastoreState) null, 
-->          localSwapDatastoreWritable = <unset>, 
-->          configOption = (vim.vm.ConfigOption) null, 
-->          vNicInfo = (vim.host.VirtualNicManagerInfo) null, 
-->          network = (vim.host.NetworkInfo) null, 
-->          provPolicy = (vim.vm.ProvisioningPolicy) null, 
-->          vFlashModules = (vim.host.VFlashManager.VFlashCacheConfigInfo.VFlashModuleConfigOption) [
-->             (vim.host.VFlashManager.VFlashCacheConfigInfo.VFlashModuleConfigOption) {
-->                vFlashModule = "vfc", 
-->                vFlashModuleVersion = "1.0.0.0", 
-->                minSupportedModuleVersion = "1.0.0.0", 
-->                cacheConsistencyType = (vim.option.ChoiceOption) {
-->                   valueIsReadonly = true, 
-->                   choiceInfo = (vim.ElementDescription) [
-->                      (vim.ElementDescription) {
-->                         label = "strong", 
-->                         summary = "Cache data consistency is guaranteed after a crash", 
-->                         key = "strong"
-->                      }
-->                   ], 
-->                   defaultIndex = 0
-->                }, 
-->                cacheMode = (vim.option.ChoiceOption) {
-->                   valueIsReadonly = true, 
-->                   choiceInfo = (vim.ElementDescription) [
-->                      (vim.ElementDescription) {
-->                         label = "write_thru", 
-->                         summary = "Writes to the cache cause writes to the underlying storage", 
-->                         key = "write_thru"
-->                      }
-->                   ], 
-->                   defaultIndex = 0
-->                }, 
-->                blockSizeInKBOption = (vim.option.LongOption) {
-->                   valueIsReadonly = <unset>, 
-->                   min = 4, 
-->                   max = 1024, 
-->                   defaultValue = 8
-->                }, 
-->                reservationInMBOption = (vim.option.LongOption) {
-->                   valueIsReadonly = <unset>, 
-->                   min = 4, 
-->                   max = 409600, 
-->                   defaultValue = 4
-->                }, 
-->                maxDiskSizeInKB = 17179869184
-->             }
-->          ], 
-->          vFlashResourceAccessible = false, 
-->       }, 
-->       primaryDatastore = (vpx.vmprov.DatastoreState) {
-->          datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->          summary = (vim.Datastore.Summary) {
-->             datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->             name = "vsanDatastore", 
-->             url = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/", 
-->             capacity = 2970573864960, 
-->             freeSpace = 2656504381440, 
-->             uncommitted = 34617360384, 
-->             accessible = true, 
-->             multipleHostAccess = true, 
-->             type = "vsan", 
-->             maintenanceMode = "normal"
-->          }, 
-->          capability = (vim.Datastore.Capability) {
-->             directoryHierarchySupported = true, 
-->             rawDiskMappingsSupported = true, 
-->             perFileThinProvisioningSupported = true, 
-->             storageIORMSupported = false, 
-->             nativeSnapshotSupported = false, 
-->             nativeSnapshotCopySupported = false, 
-->             topLevelDirectoryCreateSupported = false, 
-->             seSparseSupported = false
-->          }
-->       }, 
-->       datastore = (vpx.vmprov.DatastoreState) [
-->          (vpx.vmprov.DatastoreState) {
-->             datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->             summary = (vim.Datastore.Summary) {
-->                datastore = 'vim.Datastore:dc112ee5-1570-4c87-8d33-60c3fd66f00c:datastore-252', 
-->                name = "vsanDatastore", 
-->                url = "ds:///vmfs/volumes/vsan:52444d9ba9979afd-b48e9debd92a0c9a/", 
-->                capacity = 2970573864960, 
-->                freeSpace = 2656504381440, 
-->                uncommitted = 34617360384, 
-->                accessible = true, 
-->                multipleHostAccess = true, 
-->                type = "vsan", 
-->                maintenanceMode = "normal"
-->             }, 
-->             capability = (vim.Datastore.Capability) {
-->                directoryHierarchySupported = true, 
-->                rawDiskMappingsSupported = true, 
-->                perFileThinProvisioningSupported = true, 
-->                storageIORMSupported = false, 
-->                nativeSnapshotSupported = false, 
-->                nativeSnapshotCopySupported = false, 
-->                topLevelDirectoryCreateSupported = false, 
-->                seSparseSupported = false
-->             }
-->          }
-->       ]
-->    }, 
-->    spec = (vim.vm.RelocateSpec) {
-->       service = (vim.ServiceLocator) null, 
-->       folder = <unset>, 
-->       datastore = <unset>, 
-->       diskMoveType = <unset>, 
-->       pool = <unset>, 
-->       host = 'vim.HostSystem:dc112ee5-1570-4c87-8d33-60c3fd66f00c:host-15', 
-->       transform = <unset>, 
-->       profile = (vim.vm.ProfileSpec) [
-->          (vim.vm.DefinedProfileSpec) {
-->             profileId = "aa6d5a82-1c88-45da-85d3-3d74b91a5bad", 
-->             profileData = (vim.vm.ProfileRawData) {
-->                extensionKey = "com.vmware.vim.sps", 
-->                objectData = "<ns1:storageProfile xmlns:ns1="http://profile.policy.data.vasa.vim.vmware.com/xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns1:StorageProfile"><ns1:constraints/><ns1:createdBy>Temporary user handle</ns1:createdBy><ns1:creationTime>2016-10-05T11:44:53.323+00:00</ns1:creationTime><ns1:description>Storage policy used as default for Virtual SAN datastores</ns1:description><ns1:generationId>0</ns1:generationId><ns1:lastUpdatedBy>Temporary user handle</ns1:lastUpdatedBy><ns1:lastUpdatedTime>2016-10-05T11:44:53.323+00:00</ns1:lastUpdatedTime><ns1:name>Virtual SAN Default Storage Policy</ns1:name><ns1:profileId>aa6d5a82-1c88-45da-85d3-3d74b91a5bad</ns1:profileId></ns1:storageProfile>"
-->             }
-->          }
-->       ], 
-->    }, 
-->    priority = "defaultPriority", 
-->    drmRecommended = false, 
-->    migrationId = 1479979955112473, 
-->    changeDiskDatastore = false, 
-->    startTime = "2016-11-24T09:32:35.112474Z", 
-->    dstResult = (vim.host.VMotionManager.VMotionResult) null, 
-->    srcResult = (vim.host.VMotionManager.SrcVMotionResult) null, 
-->    numVMotionDisks = <unset>, 
--> }
2016-11-24T09:33:00.199Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.RegisterDestinationVm
2016-11-24T09:33:00.218Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.RegisterDestinationVm with output:
--> false
2016-11-24T09:33:00.220Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.SwitchHostAndPool
2016-11-24T09:33:00.220Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=VmprovUtil opID=274079b9-C2-c2-25] [PatchConfig] Not persisting the serial port network backings. If you have VMs with network backings for serial ports, please set config option: vpxd.SerialPort.PersistNetworkBacking. 
2016-11-24T09:33:00.225Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.SwitchHostAndPool with output:
--> false
2016-11-24T09:33:00.226Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.UnregisterSourceVm
2016-11-24T09:33:01.182Z warning vpxd[7FDC515D2700] [Originator@6876 sub=VmprovUtil opID=274079b9-C2-c2-25-45] [PatchConfig] Not persisting the serial port network backings. If you have VMs with network backings for serial ports, please set config option: vpxd.SerialPort.PersistNetworkBacking. 
2016-11-24T09:33:01.227Z error vpxd[7FDC515D2700] [Originator@6876 sub=corehostSync opID=274079b9-C2-c2-25-45] [Vds::Sync::UpdatePortConnecteeInt] The vm the port [590] belongs to is being migrated , we cannot change its proxyHost
2016-11-24T09:33:01.230Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.UnregisterSourceVm with output:
--> false
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.InvokeCallbacks
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on AutoStartManager with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on ContextValidator with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on DAS with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on DVS with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.232Z info vpxd[7FDC50FC6700] [Originator@6876 sub=coreanchor opID=274079b9-C2-c2-25] [MoDVSwitch::UpdatePortProxyHostInt] updating port [590] proxy host to [host-10]
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on EventEmitter with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on FT with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on InvtId with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on Iofilter with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on MetadataHandler with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on PbmCallBack with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: PbmFunctionTracer: Entering PostMigrateCallback
2016-11-24T09:33:01.273Z warning vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: ShouldSkipPostMigrateCallback: post migrate callback is skipped - VM migration failed
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=pbm opID=274079b9-C2-c2-25] PBMCallback: ~PbmFunctionTracer: Leaving  PostMigrateCallback
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on PermissionsHandler with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [PostMigrateCallback]Hit a fault so not applying permissions
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on SDRS with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on VApp with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on DRM with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on ResourceReservation with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on OCM with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [CallbackManager] Invoke post callback on StorageListeners with fault = vmodl.fault.InvalidArgument
2016-11-24T09:33:01.273Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.InvokeCallbacks with output:
--> false
2016-11-24T09:33:01.275Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Undo action vpx.vmprov.ReserveDirectory
2016-11-24T09:33:01.275Z info vpxd[7FDC50FC6700] [Originator@6876 sub=VmProv opID=274079b9-C2-c2-25] [WorkflowXAction] Done undo action vpx.vmprov.ReserveDirectory with output:
--> false
2016-11-24T09:33:01.293Z error vpxd[7FDC50FC6700] [Originator@6876 sub=vpxLro opID=274079b9-C2-c2-25] [VpxLRO] Unexpected Exception: vmodl.fault.InvalidArgument
2016-11-24T09:33:01.293Z info vpxd[7FDC50FC6700] [Originator@6876 sub=vpxLro opID=274079b9-C2-c2-25] [VpxLRO] -- FINISH task-internal-1414746
2016-11-24T09:33:01.293Z info vpxd[7FDC50FC6700] [Originator@6876 sub=Default opID=274079b9-C2-c2-25] [VpxLRO] -- ERROR task-internal-1414746 --  -- VmprovWorkflow: vmodl.fault.InvalidArgument:
--> Result:
--> (vmodl.fault.InvalidArgument) {
-->    faultCause = (vmodl.MethodFault) null, 
-->    invalidProperty = "path", 
-->    msg = ""
--> }
--> Args:
--> 

container-logs (29).zip

hickeng commented 7 years ago

Opened https://bugzilla.eng.vmware.com/show_bug.cgi?id=1772589 for internal query.

mhagen-vmware commented 7 years ago

Hit this issue on stress with the log collection turned off:

ERRO[2016-12-01T03:01:41-06:00] Failed to power on appliance A specified parameter was not correct: path. Exiting... 
ERRO[2016-12-01T03:01:41-06:00] --------------------                         
ERRO[2016-12-01T03:01:41-06:00] vic-machine-linux failed: Failed to power on appliance A specified parameter was not correct: path. Exiting...
mhagen-vmware commented 7 years ago

Currently a medium priority, this is an old bug uncommented upon because longevity/stress setup has been broken for the last 2 months. It is fixed now, and I know how to fix it moving forward so we can tackle this issue again.

mhagen-vmware commented 7 years ago

container-logs.zip

Hit this issue again upon running longevity

mhagen-vmware commented 7 years ago

@emlin longevity system is currently in this state right now. @jzt is looking at it in conjunction with the fix for the other longevity failure that he was working on. Please work together to make sure longevity is fixed as quickly as possible. If you need anything from let me know ASAP, so we can get longevity/stress unblocked.