search

vmware / terraform-provider-nsxt

Terraform Provider for VMware NSX
https://registry.terraform.io/providers/vmware/nsxt/
Mozilla Public License 2.0
131 stars 84 forks source link

Cannot use nsxt_policy_vtep_ha_host_switch_profile in nsxt_policy_host_transport_node_profile #1441

Open martinrohrbach opened 6 days ago

martinrohrbach commented 6 days ago

Describe the bug

We have adopted the nsxt_policy_host_transport_node_profile in our environment and after some initial problems (see my last issues ;) we are quite happy with that.

We are now looking into using the nsxt_policy_vtep_ha_host_switch_profile for our transport nodes. We've tested the profiles manually in our test environment by adding the VTEP HA profiles using the API and that worked fine. However, when we tried using the vTEP HA profile in the nsxt_policy_host_transport_node_profile resource, the provider fails:

│
│ Error:  Failed to update Policy Host Transport Node Profile 199f39ef-9406-4a69-b9a4-6c8876573ea5: Unable to find UplinkProfile associated with Id BaseHostSwitchProfile/ac3d4c95-7500-4062-8dd0-b0e2233908cd. (code 8500)
│
│   with nsxt_policy_host_transport_node_profile.trf-tnp["cluster1"],
│   on [transport-node-profiles.tf](http://transport-node-profiles.tf/) line 109, in resource "nsxt_policy_host_transport_node_profile" "trf-tnp":
│  109: resource "nsxt_policy_host_transport_node_profile" "trf-tnp" {
│

We've simply added the profile to the host_switch_profile property of the resource, but apparently that expects only "BaseHostSwitchProfiles" rather than VTEP HA ones.

As I currently don't see a way to add the latter, is this possibly something that can easily be added (either by accepting them in host_switch_profile or by adding an additional property)?

Or is it already possible and we overlooked something?

Reproduction steps

  1. Create nsxt_policy_vtep_ha_host_switch_profile resource
  2. Use the resouce as a profile for nsxt_policy_host_transport_node_profile.host_switch_profile
  3. Provider fails to create transport node profile (see above)

Expected behavior

We can use the VTEP HA profile to create a host transport profile.

Additional context

No response

ksamoray commented 6 days ago

Hi @martinrohrbach, Can you please include the following info:

martinrohrbach commented 6 days ago

Sure: Terraform 1.9.5

NSX-T Provider 3.6.2

NSX-T 4.1.2.5

Here's the log:

2024-10-21T12:16:14.470+0200 [INFO]  backend/local: apply calling Apply

2024-10-21T12:16:14.473+0200 [INFO]  provider: configuring client automatic mTLS

2024-10-21T12:16:14.491+0200 [INFO]  provider.terraform-provider-vsphere_v2.9.2_x5: configuring server automatic mTLS: timestamp="2024-10-21T12:16:14.491+0200"

2024-10-21T12:16:14.519+0200 [INFO]  provider: configuring client automatic mTLS

2024-10-21T12:16:14.521+0200 [WARN]  ValidateProviderConfig from "provider[\"[registry.terraform.io/hashicorp/vsphere\](http://registry.terraform.io/hashicorp/vsphere%5C)"]" changed the config value, but that value is unused

2024-10-21T12:16:14.533+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: configuring server automatic mTLS: timestamp="2024-10-21T12:16:14.533+0200"

2024-10-21T12:16:14.581+0200 [INFO]  provider.terraform-provider-vsphere_v2.9.2_x5: 2024/10/21 12:16:14 [DEBUG] Cached SOAP client session data not valid or persistence not enabled, new session necessary: timestamp="2024-10-21T12:16:14.581+0200"

2024-10-21T12:16:14.581+0200 [INFO]  provider.terraform-provider-vsphere_v2.9.2_x5: 2024/10/21 12:16:14 [DEBUG] Creating new SOAP API session on endpoint vcenter1: timestamp="2024-10-21T12:16:14.581+0200"

2024-10-21T12:16:14.582+0200 [WARN]  ValidateProviderConfig from "provider[\"[registry.terraform.io/vmware/nsxt\](http://registry.terraform.io/vmware/nsxt%5C)"]" changed the config value, but that value is unused

2024-10-21T12:16:14.595+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 [INFO]: Session headers configured for policy objects: timestamp="2024-10-21T12:16:14.595+0200"

2024-10-21T12:16:14.595+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 [INFO]: Session headers configured for policy objects: timestamp="2024-10-21T12:16:14.595+0200"

2024-10-21T12:16:14.595+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 SecurityContext schemeID is: %!(EXTRA *string=0xc000834a30): timestamp="2024-10-21T12:16:14.595+0200"

2024-10-21T12:16:14.595+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 Invoking action: "GET" and url: "https://nsxm00/api/v1/node/version": timestamp="2024-10-21T12:16:14.595+0200"

2024-10-21T12:16:14.596+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 Issuing request towards NSX:

GET /api/v1/node/version HTTP/1.1

Host: nsxm00

User-Agent: vAPI/0.7.0 Go/go1.19.13 (linux; amd64)

Content-Length: 2

<Omitted Authorization header>

Content-Type: application/json

Cookie: JSESSIONID=47EE6028EC0A5795296585DF71EAB549;

Vapi-Ctx-Opid: 1704551e-1602-40f6-a297-d63291192e46

X-Xsrf-Token: fa23d759-094a-4f6d-92e8-15de86ba9ec4

Accept-Encoding: gzip

{}: timestamp="2024-10-21T12:16:14.595+0200"

2024-10-21T12:16:14.638+0200 [INFO]  provider.terraform-provider-vsphere_v2.9.2_x5: 2024/10/21 12:16:14 [DEBUG] SOAP API session creation successful: timestamp="2024-10-21T12:16:14.638+0200"

2024-10-21T12:16:14.638+0200 [INFO]  provider.terraform-provider-vsphere_v2.9.2_x5: 2024/10/21 12:16:14 [DEBUG] VMWare vSphere Client configured for URL: vcenter1: timestamp="2024-10-21T12:16:14.638+0200"

2024-10-21T12:16:14.638+0200 [INFO]  provider.terraform-provider-vsphere_v2.9.2_x5: 2024/10/21 12:16:14 [DEBUG] Setting up REST client: timestamp="2024-10-21T12:16:14.638+0200"

2024-10-21T12:16:14.678+0200 [INFO]  provider.terraform-provider-vsphere_v2.9.2_x5: 2024/10/21 12:16:14 [DEBUG] CIS REST client configuration successful: timestamp="2024-10-21T12:16:14.678+0200"

2024-10-21T12:16:14.685+0200 [INFO]  provider: plugin process exited: plugin=.terraform/providers/[registry.terraform.io/hashicorp/vsphere/2.9.2/linux_amd64/terraform-provider-vsphere_v2.9.2_x5](http://registry.terraform.io/hashicorp/vsphere/2.9.2/linux_amd64/terraform-provider-vsphere_v2.9.2_x5) id=16822

2024-10-21T12:16:14.920+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 Received NSX response:

HTTP/1.1 200 OK

Content-Length: 85

Content-Type: application/json

Date: Mon, 21 Oct 2024 10:16:14 GMT

Server: envoy

Strict-Transport-Security: max-age=31536000; includeSubDomains

Vmw-Task-Id: fc811642-a34e-c77d-8987-bb8045f47aa2_3f7c7c6e-8a37-44c7-b2b5-c6062e019ce5

X-Envoy-Upstream-Service-Time: 312

X-Frame-Options: SAMEORIGIN

X-Xss-Protection: 1; mode=block

{

  "node_version": "4.1.2.5.0.24150847",

  "product_version": "4.1.2.5.0.24150840"

}: timestamp="2024-10-21T12:16:14.920+0200"

2024-10-21T12:16:14.920+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 [DEBUG] NSX version is 4.1.2.5.0.24150847: timestamp="2024-10-21T12:16:14.920+0200"

2024-10-21T12:16:14.936+0200 [WARN]  Provider "[registry.terraform.io/vmware/nsxt](http://registry.terraform.io/vmware/nsxt)" produced an invalid plan for nsxt_policy_host_transport_node_profile.trf-tnp["trf-tnp"], but we are tolerating it because it is using the legacy plugin SDK.

    The following problems may be the cause of any confusing errors from downstream operations:

      - .ignore_overridden_hosts: planned value cty.False for a non-computed attribute

      - .standard_host_switch[0].host_switch_mode: planned value cty.StringVal("STANDARD") for a non-computed attribute

      - .standard_host_switch[0].transport_node_profile_sub_config[0].host_switch_config_option[0].ip_assignment[0].assigned_by_dhcp: planned value cty.False for a non-computed attribute

      - .standard_host_switch[0].transport_node_profile_sub_config[1].host_switch_config_option[0].ip_assignment[0].assigned_by_dhcp: planned value cty.False for a non-computed attribute

nsxt_policy_host_transport_node_profile.trf-tnp["trf-tnp"]: Creating...

2024-10-21T12:16:14.936+0200 [INFO]  Starting apply for nsxt_policy_host_transport_node_profile.trf-tnp["trf-tnp"]

2024-10-21T12:16:14.940+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 [DEBUG] setting computed for "standard_host_switch.0.transport_zone_endpoint.0.transport_zone_profiles" from ComputedKeys: timestamp="2024-10-21T12:16:14.939+0200"

2024-10-21T12:16:14.940+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 [DEBUG] setting computed for "standard_host_switch.0.transport_zone_endpoint.0.transport_zone_profiles" from ComputedKeys: timestamp="2024-10-21T12:16:14.940+0200"

2024-10-21T12:16:14.940+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 [DEBUG] setting computed for "standard_host_switch.0.transport_zone_endpoint.0.transport_zone_profiles" from ComputedKeys: timestamp="2024-10-21T12:16:14.940+0200"

2024-10-21T12:16:14.942+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 [INFO]: Session headers configured for policy objects: timestamp="2024-10-21T12:16:14.942+0200"

2024-10-21T12:16:14.942+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 [INFO]: Session headers configured for policy objects: timestamp="2024-10-21T12:16:14.942+0200"

2024-10-21T12:16:14.942+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 SecurityContext schemeID is: %!(EXTRA *string=0xc00097f390): timestamp="2024-10-21T12:16:14.942+0200"

2024-10-21T12:16:14.943+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 Invoking action: "PUT" and url: "https://nsxm00/policy/api/v1/infra/host-transport-node-profiles/c34b7d7d-359a-4853-be38-6e1b5a7c4897": timestamp="2024-10-21T12:16:14.943+0200"

2024-10-21T12:16:14.943+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 Issuing request towards NSX:

PUT /policy/api/v1/infra/host-transport-node-profiles/c34b7d7d-359a-4853-be38-6e1b5a7c4897 HTTP/1.1

Host: nsxm00

User-Agent: vAPI/0.7.0 Go/go1.19.13 (linux; amd64)

Content-Length: 2142

<Omitted Authorization header>

Content-Type: application/json

Cookie: JSESSIONID=47EE6028EC0A5795296585DF71EAB549;

Vapi-Ctx-Opid: 7881a4ab-941a-40a0-b8cd-d1e9c85db494

X-Xsrf-Token: fa23d759-094a-4f6d-92e8-15de86ba9ec4

Accept-Encoding: gzip

{"description":"","display_name":"trf-tnp","host_switch_spec":{"host_switches":[{"host_switch_id":"50 13 5a 8d 3f 48 8e a2-77 3e 94 53 0f b8 eb b9","host_switch_mode":"STANDARD","host_switch_name":"switch-name","host_switch_profile_ids":[{"key":"UplinkHostSwitchProfile","value":"/infra/host-switch-profiles/06e26216-8729-4f8e-b6c2-1360754831fd"},{"key":"UplinkHostSwitchProfile","value":"/infra/host-switch-profiles/9f1edff0-63b4-40b1-9306-3b9fe03ff44a"}],"host_switch_type":"VDS","ip_assignment_spec":{"resource_type":"AssignedByDhcp"},"is_migrate_pnics":false,"transport_node_profile_sub_configs":[{"host_switch_config_option":{"host_switch_id":"50 13 5a 8d 3f 48 8e a2-77 3e 94 53 0f b8 eb b9","host_switch_profile_ids":[{"key":"UplinkHostSwitchProfile","value":"/infra/host-switch-profiles/06e26216-8729-4f8e-b6c2-1360754831fd"}],"ip_assignment_spec":{"ip_pool_id":"/infra/ip-pools/6d511cb8-334c-42a9-aa84-3df5845557f3","resource_type":"StaticIpPoolSpec"},"uplinks":[{"uplink_name":"uplink-1","vds_lag_name":"","vds_uplink_name":"uplink-1"},{"uplink_name":"uplink-2","vds_lag_name":"","vds_uplink_name":"uplink-2"}]},"name":"stnp-profile-1"},{"host_switch_config_option":{"host_switch_id":"50 13 5a 8d 3f 48 8e a2-77 3e 94 53 0f b8 eb b9","host_switch_profile_ids":[{"key":"UplinkHostSwitchProfile","value":"/infra/host-switch-profiles/06e26216-8729-4f8e-b6c2-1360754831fd"}],"ip_assignment_spec":{"ip_pool_id":"/infra/ip-pools/6d511cb8-334c-42a9-aa84-3df5845557f3","resource_type":"StaticIpPoolSpec"},"uplinks":[{"uplink_name":"uplink-1","vds_lag_name":"","vds_uplink_name":"uplink-1"},{"uplink_name":"uplink-2","vds_lag_name":"","vds_uplink_name":"uplink-2"}]},"name":"stnp-profile-2"}],"transport_zone_endpoints":[{"transport_zone_id":"/infra/sites/default/enforcement-points/default/transport-zones/6aa0df62-1826-40c8-9bc8-e929d0aedcc3"}],"uplinks":[{"uplink_name":"uplink-1","vds_lag_name":"","vds_uplink_name":"uplink-1"},{"uplink_name":"uplink-2","vds_lag_name":"","vds_uplink_name":"uplink-2"}]}],"resource_type":"StandardHostSwitchSpec"},"ignore_overridden_hosts":false,"tags":[]}: timestamp="2024-10-21T12:16:14.943+0200"

2024-10-21T12:16:14.972+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:14 Received NSX response:

HTTP/1.1 400 Bad Request

Transfer-Encoding: chunked

Cache-Control: no-cache, no-store, max-age=0, must-revalidate

Content-Type: application/json

Date: Mon, 21 Oct 2024 10:16:14 GMT

Expires: 0

Pragma: no-cache

Server: envoy

Strict-Transport-Security: max-age=31536000; includeSubDomains

Vary: Accept-Encoding

X-Content-Type-Options: nosniff

X-Envoy-Upstream-Service-Time: 23

X-Frame-Options: SAMEORIGIN

X-Nsx-Requestid: 591d6116-47f4-4872-a2f0-a09b712cb7d1

X-Xss-Protection: 1; mode=block

e6

{

  "httpStatus" : "BAD_REQUEST",

  "error_code" : 8500,

  "module_name" : "NsxSwitching service",

  "error_message" : "Unable to find UplinkProfile associated with Id BaseHostSwitchProfile/e83219a4-029a-486e-876e-5981a87c6d4c."

}

0

: timestamp="2024-10-21T12:16:14.972+0200"

(ommitted the "Retrying request due to error code 400")

2024-10-21T12:16:16.051+0200 [INFO]  provider.terraform-provider-nsxt_v3.6.2: 2024/10/21 12:16:16 [ERROR]:  Failed to create Policy Host Transport Node Profile c34b7d7d-359a-4853-be38-6e1b5a7c4897: Unable to find UplinkProfile associated with Id BaseHostSwitchProfile/e83219a4-029a-486e-876e-5981a87c6d4c. (code 8500): timestamp="2024-10-21T12:16:16.051+0200"

2024-10-21T12:16:16.051+0200 [ERROR] provider.terraform-provider-nsxt_v3.6.2: Response contains error diagnostic: tf_rpc=ApplyResourceChange diagnostic_summary=" Failed to create Policy Host Transport Node Profile c34b7d7d-359a-4853-be38-6e1b5a7c4897: Unable to find UplinkProfile associated with Id BaseHostSwitchProfile/e83219a4-029a-486e-876e-5981a87c6d4c. (code 8500)" tf_provider_addr=provider tf_req_id=c692765f-9f08-6403-71f6-d37d95149019 tf_resource_type=nsxt_policy_host_transport_node_profile diagnostic_detail="" diagnostic_severity=ERROR tf_proto_version=5.4 @caller=[github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/diag/diagnostics.go:58](http://github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/diag/diagnostics.go:58) @module=sdk.proto timestamp="2024-10-21T12:16:16.051+0200"

2024-10-21T12:16:16.054+0200 [ERROR] vertex "nsxt_policy_host_transport_node_profile.trf-tnp[\"trf-tnp\"]" error:  Failed to create Policy Host Transport Node Profile c34b7d7d-359a-4853-be38-6e1b5a7c4897: Unable to find UplinkProfile associated with Id BaseHostSwitchProfile/e83219a4-029a-486e-876e-5981a87c6d4c. (code 8500)

╷

│ Error:  Failed to create Policy Host Transport Node Profile c34b7d7d-359a-4853-be38-6e1b5a7c4897: Unable to find UplinkProfile associated with Id BaseHostSwitchProfile/e83219a4-029a-486e-876e-5981a87c6d4c. (code 8500)

│

│   with nsxt_policy_host_transport_node_profile.trf-tnp["trf-tnp"],

│   on [tnp.tf](http://tnp.tf/) line 62, in resource "nsxt_policy_host_transport_node_profile" "trf-tnp":

│   62: resource "nsxt_policy_host_transport_node_profile" "trf-tnp" {

│

╵

2024-10-21T12:16:16.066+0200 [INFO]  provider: plugin process exited: plugin=.terraform/providers/[registry.terraform.io/vmware/nsxt/3.6.2/linux_amd64/terraform-provider-nsxt_v3.6.2](http://registry.terraform.io/vmware/nsxt/3.6.2/linux_amd64/terraform-provider-nsxt_v3.6.2) id=16832

This is the profile that was created using the VTEP resource:

GET https://{{host}}/policy/api/v1/infra/host-switch-profiles/9f1edff0-63b4-40b1-9306-3b9fe03ff44a

{

    "enabled": false,

    "failover_timeout": 5,

    "auto_recovery": true,

    "auto_recovery_initial_wait": 300,

    "auto_recovery_max_backoff": 86400,

    "resource_type": "PolicyVtepHAHostSwitchProfile",

    "id": "9f1edff0-63b4-40b1-9306-3b9fe03ff44a",

    "display_name": "test",

    "description": "test",

    "tags": [],

    "path": "/infra/host-switch-profiles/9f1edff0-63b4-40b1-9306-3b9fe03ff44a",

    "relative_path": "9f1edff0-63b4-40b1-9306-3b9fe03ff44a",

    "parent_path": "/infra",

    "remote_path": "",

    "unique_id": "e83219a4-029a-486e-876e-5981a87c6d4c",

    "realization_id": "e83219a4-029a-486e-876e-5981a87c6d4c",

    "owner_id": "ee48d4aa-c088-4141-b063-1d7709da3dba",

    "marked_for_delete": false,

    "overridden": false,

    "_create_time": 1729505508872,

    "_create_user": "admin",

    "_last_modified_time": 1729505508872,

    "_last_modified_user": "admin",

    "_system_owned": false,

    "_protection": "NOT_PROTECTED",

    "_revision": 0

}

And we’ve simply added it to the host transport node profile as such:

resource "nsxt_policy_host_transport_node_profile" "trf-tnp" {

  …

  standard_host_switch {

    host_switch_profile = [

        data.nsxt_policy_uplink_host_switch_profile.profile.path, # <= the uplink profile

        nsxt_policy_vtep_ha_host_switch_profile.test.path,           # <= the VTEP profile

    ]

…

}

This made sense for us because looking at the API, those profiles are indeed passed in as an array with differing HostSwitchProfileTypes (UplinkHostSwitchProfile, LldpHostSwitchProfile, NiocProfile, ExtraConfigHostSwitchProfile, VtepHAHostSwitchProfile, HighPerformanceHostSwitchProfile) so in this case it would probably just be a matter of using a different type when crafting the API call?