Creating `r/virtual_machine` with multiple PCI passthrough devices fails

wladikz commented 2 years ago

Community Guidelines

[X] I have read and agree to the HashiCorp Community Guidelines .
[X] Vote on this issue by adding a 👍 reaction to the original issue initial description to help the maintainers prioritize.
[X] Do not leave "+1" or other comments that do not add relevant information or questions.
[ ] If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Terraform

1.1

Terraform Provider

2.2.0

VMware vSphere

7.0.3

Description

if I create VM with more than 1 PCI device I receive the following error:

error reconfiguring virtual machine: error reconfiguring virtual machine: A specified parameter was not correct: deviceChange[4].device.key

I enabled debug and Ii see the following request XML with the same key value for all devices.

Affected Resources

r/vsphere_virtual_machine

Terraform Configuration

No response

Debug Output

<deviceChange xmlns:XMLSchema-instance="http://www.w3.org/2001/XMLSchema-instance" XMLSchema-instance:type="VirtualDeviceConfigSpec">
          <operation>add</operation>
          <device XMLSchema-instance:type="VirtualPCIPassthrough">
            <key>0</key>
            <backing XMLSchema-instance:type="VirtualPCIPassthroughDeviceBackingInfo">
              <deviceName/>
              <id>0000:5e:00.0</id>
              <deviceId/>
              <systemId>626059a0-e234-ce2c-a1b3-5c6f69787340</systemId>
              <vendorId>4318</vendorId>
            </backing>
          </device>
        </deviceChange>
        <deviceChange xmlns:XMLSchema-instance="http://www.w3.org/2001/XMLSchema-instance" XMLSchema-instance:type="VirtualDeviceConfigSpec">
          <operation>add</operation>
          <device XMLSchema-instance:type="VirtualPCIPassthrough">
            <key>0</key>
            <backing XMLSchema-instance:type="VirtualPCIPassthroughDeviceBackingInfo">
              <deviceName/>
              <id>0000:60:00.0</id>
              <deviceId/>
              <systemId>626059a0-e234-ce2c-a1b3-5c6f69787340</systemId>
              <vendorId>4318</vendorId>
            </backing>
          </device>
        </deviceChange>
        <deviceChange xmlns:XMLSchema-instance="http://www.w3.org/2001/XMLSchema-instance" XMLSchema-instance:type="VirtualDeviceConfigSpec">
          <operation>add</operation>
          <device XMLSchema-instance:type="VirtualPCIPassthrough">
            <key>0</key>
            <backing XMLSchema-instance:type="VirtualPCIPassthroughDeviceBackingInfo">
              <deviceName/>
              <id>0000:86:00.0</id>
              <deviceId/>
              <systemId>626059a0-e234-ce2c-a1b3-5c6f69787340</systemId>
              <vendorId>4318</vendorId>
            </backing>
          </device>
        </deviceChange>

Panic Output

No response

Expected Behavior

r/virtual_machine created with multiple PCI passthrough devices.

Actual Behavior

Fails with error.

Steps to Reproduce

No response

Environment Details

No response

Screenshots

No response

References

No response

github-actions[bot] commented 2 years ago

Hello, wladikz ! 🖐

Thank you for submitting an issue for this provider. The issue will now enter into the issue lifecycle.

If you want to contribute to this project, please review the contributing guidelines and information on submitting pull requests.

tenthirtyam commented 2 years ago

HI @wladikz - I've formatted your bug report and have labeled it accordingly.

Ryan Johnson Staff II Solutions Architect | VMware, Inc.

wladikz commented 2 years ago

thanks. hope to have fix soon. we just moved from vsphere 6.7 to 7 and got this bug only on upgraded ESXi hosts

tenthirtyam commented 2 years ago

Hi @wladikz,

The issue is placed in the Backlog / Triage milestone and will be prioritized based on 👍 reactions and resource availability.

If you are interested in working on this issue and self-contributing a fix, please review the project's contributing guidelines.

Thanks!

Ryan Johnson Staff II Solutions Architect | VMware, Inc.

tenthirtyam commented 2 years ago

Based on a quick review, it's likely that the issue is within these function. More discovery is needed.

https://github.com/hashicorp/terraform-provider-vsphere/blob/f47cd954d6aefa47fdaa56318aff28b55122bef1/vsphere/internal/virtualdevice/virtual_machine_device_subresource.go#L966-L1051

Ryan Johnson Senior Staff Solutions Architect | Product Engineering @ VMware, Inc.

wladikz commented 2 years ago

Hello,

looks like it correct place. need to check why key value is always 0. if more than 1 pci device attached key value should be incremented for each device.

Regards

wladikz commented 1 year ago

Hello, Any ETA for FIX?

linuxcrash commented 1 year ago

@tenthirtyam / @wladikz, I just hit the same issue using vSphere provider 2.2.0. Trying to assign 2 GPUs using PCI Passthrough seems not to work.

2023-05-04T12:30:37.8139322Z [33;1mVERBOSE: 2023-05-04T12:30:44.399Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 [DEBUG] VM "/VxRail-Datacenter/vm/VMSK8G02" found for UUID "421c6936-12e4-218a-2657-53b29194ae39": timestamp=2023-05-04T12:30:44.399Z[0m
2023-05-04T12:30:37.8140560Z [33;1mVERBOSE: 2023-05-04T12:30:44.400Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 [DEBUG] Fetching properties for VM "/VxRail-Datacenter/vm/VMSK8G02": timestamp=2023-05-04T12:30:44.400Z[0m
2023-05-04T12:30:37.8141635Z [33;1mVERBOSE: 2023-05-04T12:30:44.404Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 [DEBUG] applyDeviceChange: Applying changes: (add: *types.VirtualPCIPassthrough at key 0): timestamp=2023-05-04T12:30:44.404Z[0m
2023-05-04T12:30:37.8142926Z [33;1mVERBOSE: 2023-05-04T12:30:44.404Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 [DEBUG] applyDeviceChange: Device list before changes: ide-200,ide-201,ps2-300,pci-100,sio-400,keyboard-600,pointing-700,video-500,vmci-12000,pvscsi-1000,disk-1000-0,disk-1000-1,ethernet-0,pcipassthrough-0: timestamp=2023-05-04T12:30:44.404Z[0m
2023-05-04T12:30:37.8144238Z [33;1mVERBOSE: 2023-05-04T12:30:44.404Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 [DEBUG] applyDeviceChange: Device list after changes: ide-200,ide-201,ps2-300,pci-100,sio-400,keyboard-600,pointing-700,video-500,vmci-12000,pvscsi-1000,disk-1000-0,disk-1000-1,ethernet-0,pcipassthrough-0,pcipassthrough-0: timestamp=2023-05-04T12:30:44.404Z[0m
2023-05-04T12:30:37.8145309Z [33;1mVERBOSE: 2023-05-04T12:30:44.404Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 VirtualMachine: PCI passthrough device specs created: timestamp=2023-05-04T12:30:44.404Z[0m
2023-05-04T12:30:37.8146197Z [33;1mVERBOSE: 2023-05-04T12:30:44.404Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 VirtualMachine: Creating PCI passthrough device specs remove: timestamp=2023-05-04T12:30:44.404Z[0m
2023-05-04T12:30:37.8147014Z [33;1mVERBOSE: 2023-05-04T12:30:44.405Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 VirtualMachine: PCI passthrough device specs created: timestamp=2023-05-04T12:30:44.404Z[0m
2023-05-04T12:30:37.8148140Z [33;1mVERBOSE: 2023-05-04T12:30:44.406Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 [DEBUG] vsphere_virtual_machine (ID = 421c6936-12e4-218a-2657-53b29194ae39): Final device list: ide-200,ide-201,ps2-300,pci-100,sio-400,keyboard-600,pointing-700,video-500,vmci-12000,pvscsi-1000,disk-1000-0,disk-1000-1,ethernet-0,pcipassthrough-0,pcipassthrough-0: timestamp=2023-05-04T12:30:44.406Z[0m
2023-05-04T12:30:37.8149526Z [33;1mVERBOSE: 2023-05-04T12:30:44.406Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 [DEBUG] vsphere_virtual_machine (ID = 421c6936-12e4-218a-2657-53b29194ae39): Final device change spec: (add: *types.VirtualPCIPassthrough at key 0),(add: *types.VirtualPCIPassthrough at key 0): timestamp=2023-05-04T12:30:44.406Z[0m
2023-05-04T12:30:37.8150566Z [33;1mVERBOSE: 2023-05-04T12:30:44.406Z [INFO]  provider.terraform-provider-vsphere_v2.2.0_x5: 2023/05/04 12:30:44 [DEBUG] Reconfiguring virtual machine "/VxRail-Datacenter/vm/VMSK8G02": timestamp=2023-05-04T12:30:44.406Z[0m
2023-05-04T12:30:37.8151312Z [33;1mVERBOSE: 2023-05-04T12:30:44.441Z [ERROR] vertex "module.vm[\"4\"].vsphere_virtual_machine.vm[0]" error: A specified parameter was not correct: deviceChange[1].device.key[0m

This looks like the PCI device key is always pointing to pcipassthrough-0 instead of incrementing it.

Is there any ETA when this could be fixed?

tenthirtyam commented 1 year ago

Is there any ETA when this could be fixed?

Hi @linuxcrash,

There's no ETA for this issue. At this time, other areas are being prioritized by the maintainers and our (VMware) collaborators depending on availability. Please refer to the milestone and labels attached to the issue for future updates.

If you'd like to submit a pull request with a fix, please review the contributing guidelines.

Ryan

linuxcrash commented 1 year ago

@tenthirtyam, Thanks for the heads up. Will manage it outside of TF then since I'm not fluent enough in GO to produce a PR.

Thanks.

spacegospod commented 5 months ago

@wladikz @linuxcrash @tenthirtyam I tried this myself and I got the same error. We usually set negative device keys for new devices which prompts vCenter to assign an actual positive number post-creation.

Adding something as simple as Key: c.VirtualDevice.NewKey() when populating the VirtualDevice structure seems to solve the problem.

Unfortunately I can't get my hands on proper hardware to test my fix yet.

github-actions[bot] commented 3 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-vsphere