search

vmware / terraform-provider-vcd

Terraform VMware Cloud Director provider
https://www.terraform.io/docs/providers/vcd/
Mozilla Public License 2.0
151 stars 111 forks source link

BUG: Update of imported `vcd_org` overwrites LDAP configuration #672

Closed bogi0704 closed 1 year ago

bogi0704 commented 3 years ago

Hi there,

After importing an existing vcd_org into terraform the "apply" fails.

Terraform Version

Terraform v0.15.4 on linux_amd64

Affected Resource(s)

Terraform Configuration Files

I have created custom Terraform modules. 

module "***-vcd-org" {
  source = "***"

  ## vcd org ##
  org_name              = var.org_name
  org_description       = "description"
  org_full_name         = "full_name"
}

Debug Output

go-vcloud-director.log: https://gist.github.com/bogi0704/155cafb68e6bcbfc609aeab49b73b638.js

Terraform (Shell) Output:

module.***-vcd-org.vcd_org.org: Modifying... [id=urn:vcloud:org:11327aa4-5cc5-476b-80f4-cb7ed9f75e88]
╷
│ Error: error updating Org error updating Org: API Error: 400: [ f289b47a-3d19-4069-9e4d-9979a0b348ed ] HTTP 400 Bad Request
│  - cvc-complex-type.2.4.a: Invalid content was found starting with element 'BackLinkIdentifier'. One of '{"http://www.vmware.com/vcloud/v1.5":MembershipIdentifier}' is expected.
│
│   with module.***-vcd-org.vcd_org.org,
│   on .terraform/modules/***-vcd-org/modules/***-vcd-org/main.tf line 1, in resource "vcd_org" "org":
│    1: resource "vcd_org" "org" {
│
╵

Expected Behavior

The planned changes (change the description of the Organization) should have been applied.

Actual Behavior

Terraform errored out.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

  1. import an existing vcd_org into your terraform state. -> successful
  2. 'terraform plan' -> successful
  3. terraform apply -> fails
Didainius commented 3 years ago

Hello @bogi0704 , Could you minimize the example to the one without modules and share a small Terraform snippet with all the configuration (you can abstract names)? Also - the log looks to be something else. Could you also share what is VCD version you are using?

Thanks

bogi0704 commented 3 years ago

Hi @Didainius,

thanks for looking into this!

terraform import vcd_org.my-org org
resource "vcd_org" "my-org" {
  name             = "org"
  full_name        = "full_name"
  description      = "'desc' - Terraform managed"
  is_enabled       = "true"
  delete_recursive = "true"
  delete_force     = "true"

  vapp_lease {
    maximum_runtime_lease_in_sec          = 0  # 1 hour
    power_off_on_runtime_lease_expiration = false
    maximum_storage_lease_in_sec          = 0     # never expires
    delete_on_storage_lease_expiration    = false
  }
  vapp_template_lease {
    maximum_storage_lease_in_sec          = 0 # 1 week
    delete_on_storage_lease_expiration    = false
  }

  can_publish_catalogs = false
}

Output:

  # vcd_org.my-org will be updated in-place
  ~ resource "vcd_org" "my-org" {
      + delete_force                 = true
      + delete_recursive             = true
      ~ description                  = "'desc'" -> "'desc' - Terraform managed"
        id                           = "urn:vcloud:org:11327aa4-5cc5-476b-80f4-cb7ed9f75e88"
        name                         = "org"
        # (6 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

vcd_org.my-org: Modifying... [id=urn:vcloud:org:11327aa4-5cc5-476b-80f4-cb7ed9f75e88]
╷
│ Error: error updating Org error updating Org: API Error: 400: [ b820a89b-26a2-4a15-90f0-0622609aafd2 ] HTTP 400 Bad Request
│  - cvc-complex-type.2.4.a: Invalid content was found starting with element 'BackLinkIdentifier'. One of '{"http://www.vmware.com/vcloud/v1.5":MembershipIdentifier}' is expected.
│
│   with vcd_org.my-org,
│   on 001.tf line 1, in resource "vcd_org" "my-org":
│    1: resource "vcd_org" "my-org" {
│
╵

The log should be the correct one. Look at the lines 124 and 125 it says Bad Request and

Invalid content was found starting with element 'BackLinkIdentifier'

aswell.

VCD is on: VMware Cloud Director version: 10.2.2.17855680 Tue Apr 06 17:43:52 UTC 2021

dataclouder commented 3 years ago

Hi, Thanks for the update. However, I could not reproduce your problem.

First, I created an Org using your script

resource "vcd_org" "my-org" {
  name = "org"
  full_name = "full_name"
  description = "'desc' - Terraform managed"
  is_enabled = "true"
  delete_recursive = "true"
  delete_force = "true"

  vapp_lease {
    maximum_runtime_lease_in_sec = 0 # 1 hour
    power_off_on_runtime_lease_expiration = false
    maximum_storage_lease_in_sec = 0 # never expires
    delete_on_storage_lease_expiration = false
  }

  vapp_template_lease {
    maximum_storage_lease_in_sec = 0 # 1 week
    delete_on_storage_lease_expiration = false
  }

  can_publish_catalogs = false
}

Then I removed it from the state and imported it

$ terraform state list
vcd_org.my-org

$ terraform state rm vcd_org.my-org

$ terraform state list

$ terraform import vcd_org.my-org org
vcd_org.my-org: Importing from ID "org"...
vcd_org.my-org: Import prepared!
  Prepared vcd_org for import
vcd_org.my-org: Refreshing state... [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

The subsequent plan and apply worked as expected

$ terraform plan
vcd_org.my-org: Refreshing state... [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vcd_org.my-org will be updated in-place
  ~ resource "vcd_org" "my-org" {
      + delete_force                 = true
      + delete_recursive             = true
        id                           = "urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f"
        name                         = "org"
        # (7 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

$ terraform apply -auto-approve
vcd_org.my-org: Refreshing state... [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]
vcd_org.my-org: Modifying... [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]
vcd_org.my-org: Modifications complete after 3s [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

This was a VCD 10.2.

bogi0704 commented 3 years ago

Please try to trigger an in-place update of the description. As soon as that is necessary it fails for me. Also the imported org has been created "by hand", maybe that has something to do with the issue I am facing.

dataclouder commented 3 years ago

No luck

$ terraform plan
vcd_org.my-org: Refreshing state... [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vcd_org.my-org will be updated in-place
  ~ resource "vcd_org" "my-org" {
      ~ description                  = "'desc' - Terraform managed" -> "new 'desc' - Terraform managed"
        id                           = "urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f"
        name                         = "org"
        # (8 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

$ terraform apply -auto-approve
vcd_org.my-org: Refreshing state... [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]
vcd_org.my-org: Modifying... [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]
vcd_org.my-org: Modifications complete after 3s [id=urn:vcloud:org:030c84f4-7ded-4dd0-8509-d63cd000399f]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
bogi0704 commented 3 years ago

Hmmm ok. The Organisation has actually been created with the vRealize Orchestrator (vRO). It looks like there are some differences between the vRO build and the Terraform one. I will report back if we find something.

bogi0704 commented 3 years ago

It looks like we have found the issue. The ORG that I was trying to import has been created via vRO. We are always creating ORGs with LDAP configured as an identity provider. It looks like the ORG is expecting some configured fields via the 'terraform apply' that I am doing after the import, in this case it is 'MembershipIdentifier'. Here is an example of our LDAP configuration.

        <OrgLdapSettings href="https://***/api/admin/org/11327aa4-5cc5-476b-80f4-cb7ed9f75e88/settings/ldap" type="application/vnd.vmware.admin.organizationLdapSettings+xml">
            <OrgLdapMode>CUSTOM</OrgLdapMode>
            <CustomOrgLdapSettings>
                <HostName>domain</HostName>
                <Port>Port</Port>
                <IsSsl>true</IsSsl>
                <IsSslAcceptAll>true</IsSslAcceptAll>
                <SearchBase>Searchbase</SearchBase>
                <UserName>User</UserName>
                <AuthenticationMechanism>SIMPLE</AuthenticationMechanism>
                <IsGroupSearchBaseEnabled>false</IsGroupSearchBaseEnabled>
                <ConnectorType>OPEN_LDAP</ConnectorType>
                <UserAttributes>
                    <ObjectClass>user</ObjectClass>
                    <ObjectIdentifier>userPrincipalName</ObjectIdentifier>
                    <UserName>userPrincipalName</UserName>
                    <Email>mail</Email>
                    <FullName>displayName</FullName>
                    <GivenName>givenName</GivenName>
                    <Surname>sn</Surname>
                    <Telephone>telephoneNumber</Telephone>
                    <GroupMembershipIdentifier>dn</GroupMembershipIdentifier>
                    <GroupBackLinkIdentifier>tokenGroups</GroupBackLinkIdentifier>
                </UserAttributes>
                <GroupAttributes>
                    <ObjectClass>group</ObjectClass>
                    <ObjectIdentifier>sAMAccountName</ObjectIdentifier>
                    <GroupName>cn</GroupName>
                    <Membership>member</Membership>
                    <MembershipIdentifier>dn</MembershipIdentifier>
                    <BackLinkIdentifier>objectSid</BackLinkIdentifier>
                </GroupAttributes>
                <UseExternalKerberos>false</UseExternalKerberos>
            </CustomOrgLdapSettings>
        </OrgLdapSettings>

Is there a way to avoid this? I did not find a way to configure an identity provider via the Terraform provider. The issue https://github.com/vmware/terraform-provider-vcd/issues/565 is about implementing LDAP as an identityprovider. https://github.com/vmware/terraform-provider-vcd/issues/565#issuecomment-844200178 This looks like a similar issue I am facing right now.

carmine73 commented 3 years ago

https://github.com/vmware/terraform-provider-vcd/issues/565#issuecomment-844200178

I have the same issue if I create the org with terraform, then I add LDAP configuration (actually using vRO). After that I cannot modify the organization using import + plan + apply

dataclouder commented 2 years ago

Using the newly implemented vcd_org_ldap resource, this problem should go away.

For example:

resource "vcd_org" "dummy" {
  name                            = "dummy"
  full_name                       = "dummy"
  is_enabled                      = "true"
  stored_vm_quota                 = 50
  deployed_vm_quota               = 50
  delete_force                    = "true"
  delete_recursive                = "true"
  can_publish_external_catalogs   = "true"
  can_subscribe_external_catalogs = "true"
  # description                     = "dummy org"

  vapp_lease {
    maximum_runtime_lease_in_sec          = 60*60*24*30 
    power_off_on_runtime_lease_expiration = false
    maximum_storage_lease_in_sec          = 0
    delete_on_storage_lease_expiration    = false
  }

  vapp_template_lease {
    maximum_storage_lease_in_sec       = 60*60*24*3
    delete_on_storage_lease_expiration = false
  }
}

resource "vcd_org_ldap" "dummy" {
  org_id    = vcd_org.dummy.id
  ldap_mode = "CUSTOM"
  custom_settings {
    server                  = "192.168.1.15"
    port                    = 389
    is_ssl                  = false
    username                = "cn=admin,dc=planetexpress,dc=com"
    password                = "GoodNewsEveryone"
    authentication_method   = "SIMPLE"
    base_distinguished_name = "dc=planetexpress,dc=com"
    connector_type          = "OPEN_LDAP"
    user_attributes {
      object_class                = "inetOrgPerson"
      unique_identifier           = "uid"
      display_name                = "cn"
      username                    = "uid"
      given_name                  = "givenName"
      surname                     = "sn"
      telephone                   = "telephoneNumber"
      group_membership_identifier = "dn"
      email                       = "mail"
    }
    group_attributes {
      name                        = "cn"
      object_class                = "group"
      membership                  = "member"
      unique_identifier           = "cn"
      group_membership_identifier = "dn"
    }
  }
  lifecycle {
    # password value does not get returned by GET
    ignore_changes = [custom_settings[0].password]
  }
}

I created this Org, with attached LDAP configuration. Then I removed the Org from terraform state (terraform state rm vcd_org.dummy), I imported it, and tried adding a description and changing the lease. Everything works as expected, using VCD 10.4.0.

bogi0704 commented 2 years ago

Thank you for implementing the ldap ressource. We have actually since moved to SAML, but I will test if I can import an organization with SAML configured without issues. Thanks for this nevertheless, I am sure it will come in handy.

carmine73 commented 1 year ago

Hi, still got error if I try to modify a org (just the description) with LDAP configured using import + plan + apply. is what you expect? plan + apply work if a use the commented description 

## source org
data "vcd_org" "my-org1" {
  name = var.input.org.name
}

## organization to be deleted (imported first)
resource "vcd_org" "org1" {
  # org should be disabled, delete_force and delete_recursive should be set to true (after import is null) but
  # org cannot be modified when LDAP is configured (issue on terraform provider)
  name                 = var.input.org.name
  full_name            = data.vcd_org.my-org1.full_name
  description          = "New description: Org to be deleted"
  #description          = data.vcd_org.my-org1.description
  is_enabled           = data.vcd_org.my-org1.is_enabled
  delete_force         = false
  delete_recursive     = false
  can_publish_catalogs = data.vcd_org.my-org1.can_publish_catalogs
}
dataclouder commented 1 year ago

The error reported at the top only appears if the LDAP configuration was incomplete or invalid (applied outside Terraform). I confirm that if you create the Org and apply LDAP using Terraform, there is no error when updating, with or without import.

However, the resource should be able to identify a faulty LDAP configuration and ignoring it rather than trying to apply it. I will look into it, but please confirm that using Terraform for both resources (vcd_org and vcd_org_ldap) solves the problem.

dataclouder commented 1 year ago

I have added a fix to the Org update: if an invalid LDAP is detected, it gets ignored. The right way of updating a LDAP settings id through vcd_org_ldap. If you need to import an org with LDAP settings, you should import both resources. Only with this double resource can Terraform keep track of the Org LDAP configuration.

carmine73 commented 1 year ago

I confirm that I configure LDAP outside terraform (vcd_org_ldap is supported in provider v3.8+ and I had to find a way to work with LDAP). Actually the LDAP configuration works, I'll investigate to understand why is detected as invalid. I'll try the fix ASAP, thanks

carmine73 commented 1 year ago

The resource vdc_org_ldap is imported with no errors, how can I check if it's invalid?

dataclouder commented 1 year ago

We check it by running a test that imports LDAP groups and users. But just looking at the "groups" tab in the "Administration" part will tell you whether the configuration is OK. If the "import groups" link appears, LDAP is working.

carmine73 commented 1 year ago

So my LDAP is working (I can see "import groups" link and I have also groups). Anyway I'll try with this new version. I don't need to update the LDAP configuration via terraform and I do not have LDAP configured for all orgs in my vcloud, so I'd like not to import vcd_org_ldap resource.

dataclouder commented 1 year ago

You can, however, use a data source for vcd_org_ldap and see the resulting configuration in the terraform state file. If you encounter the same error after this fix, I'd like to see what the state file contains for vcd_org_ldap data source.

carmine73 commented 1 year ago

I've got the same issue with current 3.8.1

From https://github.com/vmware/terraform-provider-vcd
   3720cb7b..481b5205  main       -> origin/main
Updating 3720cb7b..481b5205
Fast-forward
. . .
$ terraform version 
Terraform v1.3.6
on linux_amd64
+ provider registry.terraform.io/vmware/vcd v3.8.1

this is the plan output for the org

Terraform will perform the following actions:

  # vcd_org.org1 will be updated in-place
  ~ resource "vcd_org" "org1" {
      + delete_force                    = false
      + delete_recursive                = false
      ~ description                     = "org-nsxvcol_1 created by terraform + vmware/vcd plugin on 2022-12-01T09:55:12Z" -> "org-nsxvcol_1 created by terraform + vmware/vcd plugin on 2022-12-01T09:55:12Z - Org to be deleted"
        id                              = "urn:vcloud:org:f85f2188-7610-42c3-8b7f-f914d3613495"
        name                            = "org-nsxvcol_1"
        # (9 unchanged attributes hidden)

        # (7 unchanged blocks hidden)
    }

and this is the error

$ terraform apply "orgs/COLL/org-nsxvcol_1/org-nsxvcol_1.plan"
vcd_org.org1: Modifying... [id=urn:vcloud:org:f85f2188-7610-42c3-8b7f-f914d3613495]
╷
│ Error: error updating Org error updating Org: API Error: 400: [ 7fd1156d-38d3-4a77-b9d0-2b02ecb111f9 ] HTTP 400 Bad Request
│  - cvc-complex-type.2.4.a: Invalid content was found starting with element '{"http://www.vmware.com/vcloud/v1.5":BackLinkIdentifier}'. One of '{"http://www.vmware.com/vcloud/v1.5":MembershipIdentifier}' is expected.
│ 
│   with vcd_org.org1,
│   on sdc-org-removal.tf line 31, in resource "vcd_org" "org1":
│   31: resource "vcd_org" "org1" {
│

and this is the vcd_org_ldap I got with data source

    {
      "mode": "data",
      "type": "vcd_org_ldap",
      "name": "my-org1-ldap",
      "provider": "provider[\"registry.terraform.io/vmware/vcd\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "custom_settings": [
              {
                "authentication_method": "SIMPLE",
                "base_distinguished_name": "ou=org-nsxvcol_1,ou=organizations,dc=usersdomain,dc=domain",
                "connector_type": "ACTIVE_DIRECTORY",
                "group_attributes": [
                  {
                    "group_back_link_identifier": "objectSid",
                    "group_membership_identifier": "objectGuid",
                    "membership": "member",
                    "name": "cn",
                    "object_class": "group",
                    "unique_identifier": "dn"
                  }
                ],
                "is_ssl": true,
                "password": "",
                "port": 636,
                "server": "adhostname.usersdomain.domain",
                "user_attributes": [
                  {
                    "display_name": "displayName",
                    "email": "mail",
                    "given_name": "givenName",
                    "group_back_link_identifier": "tokenGroups",
                    "group_membership_identifier": "dn",
                    "object_class": "user",
                    "surname": "sn",
                    "telephone": "telephoneNumber",
                    "unique_identifier": "objectGuid",
                    "username": "mail"
                  }
                ],
                "username": "ldap_username"
              }
            ],
            "id": "urn:vcloud:org:f85f2188-7610-42c3-8b7f-f914d3613495",
            "ldap_mode": "CUSTOM",
            "org_id": "urn:vcloud:org:f85f2188-7610-42c3-8b7f-f914d3613495"
          },
          "sensitive_attributes": []
        }
      ]
    },

this is the PUT request on logs

2022/12/09 09:50:44 --------------------------------------------------------------------------------
2022/12/09 09:50:44 Request caller: govcd.(*AdminOrg).Update-->govcd.(*AdminOrg).Update-->govcd.executeRequestWithApiVersion-->govcd.(*Client).executeTaskRequest-->govcd.executeRequestCustomErr-->govcd.executeRequestCustomErr-->govcd.(*Client).newRequest
2022/12/09 09:50:44 PUT https://vdchost.domain/api/admin/org/f85f2188-7610-42c3-8b7f-f914d3613495
2022/12/09 09:50:44 --------------------------------------------------------------------------------
2022/12/09 09:50:44 Req header:
2022/12/09 09:50:44     X-Vmware-Vcloud-Access-Token: [********]
2022/12/09 09:50:44     Accept: [application/*+xml;version=36.0]
2022/12/09 09:50:44     X-Vmware-Vcloud-Token-Type: [Bearer]
2022/12/09 09:50:44     Authorization: [********]
2022/12/09 09:50:44     User-Agent: [terraform-provider-vcd/v3.8.0-4-g481b5205 (linux/amd64; isProvider:true)]
2022/12/09 09:50:44 Request data: [8860]
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
  <AdminOrg xmlns="http://www.vmware.com/vcloud/v1.5" name="org-nsxvcol_1">
    [. . .]
          <OrgLdapSettings href="https://vdchost.domain/api/admin/org/f85f2188-7610-42c3-8b7f-f914d3613495/settings/ldap" type="application/vnd.vmware.admin.organizationLdapSettings+xml">
              <Link href="https://vdchost.domain/api/admin/org/f85f2188-7610-42c3-8b7f-f914d3613495/settings/ldap" type="application/vnd.vmware.admin.organizationLdapSettings+xml" rel="edit"></Link>
              <Link href="https://vdchost.domain/api/admin/org/f85f2188-7610-42c3-8b7f-f914d3613495/settings/ldap" type="application/vnd.vmware.admin.organizationLdapSettings+json" rel="edit"></Link>
              <Link href="https://vdchost.domain/cloudapi/1.0.0/ldap/test" type="application/json" rel="down"></Link>
              <Link href="https://vdchost.domain/cloudapi/1.0.0/ldap/sync" type="application/json" rel="down"></Link>
              <OrgLdapMode>CUSTOM</OrgLdapMode>
              <CustomOrgLdapSettings>
                  <HostName>adhostname.usersdomain.domain</HostName>
                  <Port>636</Port>
                  <IsSsl>true</IsSsl>
                  <SearchBase>ou=org-nsxvcol_1,ou=organizations,dc=usersdomain,dc=domain</SearchBase>
                  <UserName>ldap_username</UserName>
                  <AuthenticationMechanism>SIMPLE</AuthenticationMechanism>
                  <IsGroupSearchBaseEnabled>false</IsGroupSearchBaseEnabled>
                  <ConnectorType>ACTIVE_DIRECTORY</ConnectorType>
                  <UserAttributes>
                      <ObjectClass>user</ObjectClass>
                      <ObjectIdentifier>objectGuid</ObjectIdentifier>
                      <UserName>mail</UserName>
                      <Email>mail</Email>
                      <FullName>displayName</FullName>
                      <GivenName>givenName</GivenName>
                      <Surname>sn</Surname>
                      <Telephone>telephoneNumber</Telephone>
                      <GroupMembershipIdentifier>dn</GroupMembershipIdentifier>
                      <GroupBackLinkIdentifier>tokenGroups</GroupBackLinkIdentifier>
                  </UserAttributes>
                  <GroupAttributes>
                      <ObjectClass>group</ObjectClass>
                      <ObjectIdentifier>dn</ObjectIdentifier>
                      <GroupName>cn</GroupName>
                      <Membership>member</Membership>
                      <BackLinkIdentifier>objectSid</BackLinkIdentifier>
                      <MembershipIdentifier>objectGuid</MembershipIdentifier>
                  </GroupAttributes>
                  <UseExternalKerberos>false</UseExternalKerberos>
              </CustomOrgLdapSettings>
          </OrgLdapSettings>
      </Settings>
  </AdminOrg>

and the response

2022/12/09 09:50:44 ################################################################################
2022/12/09 09:50:44 Response caller govcd.(*AdminOrg).Update-->govcd.(*AdminOrg).Update-->govcd.executeRequestWithApiVersion-->govcd.(*Client).executeTaskRequest-->govcd.executeRequestCustomErr-->govcd.checkRespWithErrType-->govcd.ParseErr-->govcd.decodeBody
2022/12/09 09:50:44 Response status 400 Bad Request
2022/12/09 09:50:44 ################################################################################
2022/12/09 09:50:44 Response header:
2022/12/09 09:50:44     Cache-Control: [no-store, must-revalidate]
2022/12/09 09:50:44     Date: [Fri, 09 Dec 2022 08:50:44 GMT]
2022/12/09 09:50:44     X-Vmware-Vcloud-Request-Id: [7fd1156d-38d3-4a77-b9d0-2b02ecb111f9]
2022/12/09 09:50:44     Content-Type: [application/vnd.vmware.vcloud.error+xml;version=36.0]
2022/12/09 09:50:44     X-Vmware-Vcloud-Request-Execution-Time: [11,11]
2022/12/09 09:50:44 Response text: [12061]
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:vmext="http://www.vmware.com/vcloud/extension/v1.5" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:common="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" xmlns:ns9="http://www.vmware.com/vcloud/versions" stackTrace="javax.ws.rs.BadRequestException: HTTP 400 Bad Request&#10;    at org.apache.cxf.jaxrs.utils.SpecExceptions.toBadRequestException(SpecExceptions.java:84)&#10; at org.apache.cxf.jaxrs.utils.ExceptionUtils.toBadRequestException(ExceptionUtils.java:120)&#10;    at org.apache.cxf.jaxrs.provider.AbstractJAXBProvider.handleExceptionEnd(AbstractJAXBProvider.java:727)&#10;    at org.apache.cxf.jaxrs.provider.AbstractJAXBProvider.handleJAXBException(AbstractJAXBProvider.java:753)&#10;   at org.apache.cxf.jaxrs.provider.JAXBElementProvider.readFrom(JAXBElementProvider.java:205)&#10;    at org.apache.cxf.jaxrs.provider.JAXBElementTypedProvider.readFrom(JAXBElementTypedProvider.java:41)&#10;   at org.apache.cxf.jaxrs.provider.JAXBElementTypedProvider.readFrom(JAXBElementTypedProvider.java:34)&#10;   at org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBodyReader(JAXRSUtils.java:1500)&#10;   at org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBody(JAXRSUtils.java:1452)&#10; at org.apache.cxf.jaxrs.utils.JAXRSUtils.processRequestBodyParameter(JAXRSUtils.java:950)&#10;  at org.apache.cxf.jaxrs.utils.JAXRSUtils.processParameters(JAXRSUtils.java:881)&#10;    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:283)&#10; at com.vmware.vcloud.api.rest.jaxrs.ContextClassLoaderInvoker.invoke(ContextClassLoaderInvoker.java:27)&#10;    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)&#10; at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)&#10;   at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)&#10;   at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)&#10;  at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)&#10;    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265)&#10;  at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)&#10;    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)&#10;   at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)&#10;   at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:225)&#10;   at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:298)&#10;    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPut(AbstractHTTPServlet.java:234)&#10;    at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)&#10;    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:273)&#10;  at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)&#10;  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)&#10; at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)&#10;    at com.vmware.vcloud.api.framework.web.ExtensibilityFilter.doFilter(ExtensibilityFilter.java:159)&#10;  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)&#10;    at com.vmware.vcloud.security.filters.ValidityExceptionFilter.doFilterHttp(ValidityExceptionFilter.java:47)&#10;    at com.vmware.vcloud.security.filters.HttpFilterBean.doFilter(HttpFilterBean.java:35)&#10;  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)&#10;    at com.vmware.vcloud.security.web.RestSessionFilter.doFilter(RestSessionFilter.java:235)&#10;   at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)&#10;    at com.vmware.vcloud.security.web.CellLivenessCheckFilter.doFilter(CellLivenessCheckFilter.java:54)&#10;    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)&#10;    at com.vmware.vcloud.security.web.SecurityContextCleanerFilter.doFilter(SecurityContextCleanerFilter.java:32)&#10;  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)&#10;    at com.vmware.vcloud.api.rest.versioning.AcceptHeaderFilter.doFilter(AcceptHeaderFilter.java:78)&#10;   at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)&#10;    at com.vmware.vcloud.security.filters.ConversationFilter$1.run(ConversationFilter.java:39)&#10; at com.vmware.vcloud.security.filters.ConversationFilter$1.run(ConversationFilter.java:37)&#10; at com.vmware.vcloud.common.persist.ConversationContextExecutor.execute(ConversationContextExecutor.java:67)&#10;   at com.vmware.vcloud.security.filters.ConversationFilter.doFilter(ConversationFilter.java:45)&#10;  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)&#10;    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)&#10;   at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)&#10;   at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)&#10; at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)&#10;   at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)&#10; at com.vmware.vcloud.common.web.deployment.impl.HostCheckerFilter.doFilter(HostCheckerFilter.java:107)&#10; at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)&#10; at com.vmware.vcloud.common.web.deployment.impl.RequestDiagnosticFilter.doFilter(RequestDiagnosticFilter.java:139)&#10; at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)&#10; at com.vmware.vcloud.common.web.deployment.impl.ConfigurableSniffyFilter.doFilter(ConfigurableSniffyFilter.java:48)&#10;    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)&#10; at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)&#10;  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)&#10;   at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)&#10; at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)&#10;   at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)&#10;  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)&#10;   at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)&#10;    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)&#10;   at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)&#10;   at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)&#10; at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)&#10;    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)&#10;   at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)&#10;  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)&#10; at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)&#10;    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)&#10; at org.eclipse.jetty.server.Server.handle(Server.java:539)&#10; at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)&#10;   at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)&#10; at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)&#10; at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)&#10;   at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:312)&#10;   at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)&#10; at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)&#10;   at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)&#10; at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)&#10;  at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)&#10; at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)&#10;    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)&#10;    at com.vmware.vcloud.common.threading.ExtendedJettyQueuedThreadPool.runJob(ExtendedJettyQueuedThreadPool.java:40)&#10;  at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)&#10; at java.base/java.lang.Thread.run(Thread.java:829)&#10;Caused by: org.xml.sax.SAXParseException; lineNumber: 74; columnNumber: 23; cvc-complex-type.2.4.a: Invalid content was found starting with element '{&quot;http://www.vmware.com/vcloud/v1.5&quot;:BackLinkIdentifier}'. One of '{&quot;http://www.vmware.com/vcloud/v1.5&quot;:MembershipIdentifier}' is expected.&#10;    at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204)&#10;  at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:135)&#10;    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)&#10;    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)&#10;    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)&#10;    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator$XSIErrorReporter.reportError(XMLSchemaValidator.java:511)&#10;    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.reportSchemaError(XMLSchemaValidator.java:3587)&#10;  at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleStartElement(XMLSchemaValidator.java:1971)&#10; at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.startElement(XMLSchemaValidator.java:829)&#10;    at java.xml/com.sun.org.apache.xerces.internal.jaxp.validation.ValidatorHandlerImpl.startElement(ValidatorHandlerImpl.java:570)&#10;    at com.sun.xml.bind.v2.runtime.unmarshaller.ValidatingUnmarshaller.startElement(ValidatingUnmarshaller.java:101)&#10;   at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.handleStartElement(StAXStreamConnector.java:246)&#10;   at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:180)&#10;   at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:415)&#10; at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:394)&#10;  at org.apache.cxf.jaxrs.provider.JAXBElementProvider.readFrom(JAXBElementProvider.java:184)&#10;    ... 86 more&#10;" message="[ 7fd1156d-38d3-4a77-b9d0-2b02ecb111f9 ] HTTP 400 Bad Request&#10; - cvc-complex-type.2.4.a: Invalid content was found starting with element '{&quot;http://www.vmware.com/vcloud/v1.5&quot;:BackLinkIdentifier}'. One of '{&quot;http://www.vmware.com/vcloud/v1.5&quot;:MembershipIdentifier}' is expected." majorErrorCode="400" minorErrorCode="BAD_REQUEST"/>
dataclouder commented 1 year ago

@carmine73 , Thanks for keeping up with me and for testing the provider before the release. I was finally able to reproduce the failure, and applied a patch. If you want to test it, add this line at the bottom of go.mod and rebuild the provider:

replace github.com/vmware/go-vcloud-director/v2 => github.com/dataclouder/go-vcloud-director/v2 v2.17.0-alpha.3.0.20221209181540-bf4af8efbe7c

Update: fix is in main branch

dataclouder commented 1 year ago

Tested using the folllowing script

resource "vcd_org" "dummy" {
  name                            = "dummy"
  full_name                       = "dummy"
  is_enabled                      = "true"
  stored_vm_quota                 = 50
  deployed_vm_quota               = 50
  delete_force                    = "true"
  delete_recursive                = "true"
  can_publish_external_catalogs   = "true"
  can_subscribe_external_catalogs = "true"
  description                     = "dummy org updated 12345"

  vapp_lease {
    maximum_runtime_lease_in_sec          = 60*60*24*30 
    power_off_on_runtime_lease_expiration = false
    maximum_storage_lease_in_sec          = 0
    delete_on_storage_lease_expiration    = false
  }

  vapp_template_lease {
    maximum_storage_lease_in_sec       = 60*60*24*3
    delete_on_storage_lease_expiration = false
  }
}
/* DELETE from here after apply */
resource "vcd_org_ldap" "dummy" {
  org_id    = vcd_org.dummy.id
  ldap_mode = "CUSTOM"
  custom_settings {
    server                  = var.ldap_ip
    port                    = 389
    is_ssl                  = false
    username                = "cn=admin,dc=planetexpress,dc=com"
    password                = "GoodNewsEveryone"
    authentication_method   = "SIMPLE"
    base_distinguished_name = "dc=planetexpress,dc=com"
    connector_type          = "OPEN_LDAP"
    user_attributes {
      object_class                = "inetOrgPerson"
      unique_identifier           = "uid"
      display_name                = "cn"
      username                    = "uid"
      given_name                  = "givenName"
      surname                     = "sn"
      telephone                   = "telephoneNumber"
      group_membership_identifier = "dn"
      email                       = "mail"
      group_back_link_identifier  = "tokenGroups"
    }
    group_attributes {
      name                        = "cn"
      object_class                = "group"
      membership                  = "member"
      unique_identifier           = "cn"
      group_membership_identifier = "dn"
      group_back_link_identifier  = "objectSid"
    }
  }
  lifecycle {
    # password value does not get returned by GET
    ignore_changes = [custom_settings[0].password]
  }
}
  1. apply the script above (change the LDAP server IP or hostname).
  2. remove terraform state file
  3. remove or comment out from the script the resource vcd_org_ldap.dummy
  4. run terraform import vcd_org.dummy dummy
  5. Change the org description and run terraform apply
  6. check in the UI that the LDAP configuration persists, and that you can import groups.
dataclouder commented 1 year ago

Fixed in PR #955

carmine73 commented 1 year ago

I wasn't able to test your fix before the merge. anyway tested using the last version of main branch

From https://github.com/vmware/terraform-provider-vcd
   481b5205..fbbfa5eb  main       -> origin/main
Updating 481b5205..fbbfa5eb
Fast-forward
 .changes/v3.8.1/952-bug-fixes.md                                   | 2 +-

it works. thanks!

Update: tested with VMware Cloud Director 10.3

dataclouder commented 1 year ago

Thanks for testing it!

carmine73 commented 1 year ago

is there a release date for 3.8.1?

dataclouder commented 1 year ago

It will be soon

dataclouder commented 1 year ago

https://github.com/vmware/terraform-provider-vcd/releases/tag/v3.8.1

carmine73 commented 1 year ago

I see, thanks!!!