Closed fgrehl closed 4 years ago
vbauzys
commented
4 years ago Hi @fgrehl, Thank you for report and analysis. I have prepared PR, could you check that will help you also https://github.com/vmware/go-vcloud-director/pull/293. It can manually tested by leaving Version the same 29.0 and adding filterEncoded=true
fgrehl
commented
4 years ago Thanks for pointing me in the right direction.
Unfortunately, just adding filterEncoded=true does not work for me. This is what I did:
git clone https://github.com/vmware/go-vcloud-director.git git fetch origin pull/293/head:pull_293 git checkout pull_293
Then I replaced the module in terraform-provider-vcd/go.mod and compiled it: replace github.com/vmware/go-vcloud-director/v2 v2.7.0-alpha.3 => ../../tmp/go-vcloud-director
Terraform still fails with the same error. I can see that it uses "filterEncoded=true" in go-vcloud-director.log.
I then modified the API Version to 31.0 and recompiled it:
go-vcloud-director/govcd/api_vcd.go: APIVersion: "31.0", // supported by vCD 9.0, 9.1, 9.5, 9.7, 10.0
With this fix, Terraform creates the oVDC successfully.
I also can still reproduce the Issue with direct API Calls:
{{ vcd_url }}/api/query?&filter=name==pVDC-XAL-METRO&type=providerVdc&filterEncoded=true ACCEPT: application/*+xml;version=29.0 EMPTY result!
{{ vcd_url }}/api/query?&filter=name==pVDC-XAL-METRO&type=providerVdc&filterEncoded=true ACCEPT: application/*+xml;version=31.0 Works!
vbauzys
commented
4 years ago @fgrehl Thank you for testing it. In our environments it was enough. Also could you verify that version=30.0 is enough for your case?
fgrehl
commented
4 years ago No, 30.0 does not work either. The pVDC query only works with 31.0/32.0/33.0.
I have 2 environments (Running identical versions) to verify at the moment, it's the same with both.
vbauzys
commented
4 years ago Thank you for testing in your env. We will search for solutions.
Didainius
commented
4 years ago @fgrehl, I have tried to run a copy of your config (even named network pool and pVdc as in your example
resource "vcd_org_vdc" "ovdc-fgr" {
name = "ovdc-fgr"
description = "My Org"
org = data.vcd_org.ddd.name
allocation_model = "AllocationPool"
network_pool_name = "VCD-Overlay-TZ"
provider_vdc_name = "PVDC-NUCluster"
compute_capacity {
cpu {
allocated = 10
}
memory {
allocated = 10
}
}
storage_profile {
name = "*"
enabled = true
limit = 0
default = true
}
enabled = true
enable_thin_provisioning = true
enable_fast_provisioning = false
delete_force = true
delete_recursive = true
}It worked for me.
My env details: vCD - 10.0.0.15449638 (looks to be exactly like yours) NSX - 2.5.0.0.0.14552524 (just a bit older)
The query log itself:
2020/03/27 10:41:39 --------------------------------------------------------------------------------
2020/03/27 10:41:39 Request caller: schema.(*Resource).Apply-->vcd.resourceVcdVdcCreate-->vcd.getVcdVdcInput-->govcd.QueryProviderVdcByName-->govcd.(*VCDClient).QueryWithNotEncodedParams-->govcd.(*Client).QueryWithNotEncodedParams
2020/03/27 10:41:39 GET https://test-host/api/query?&type=providerVdc&filterEncoded=true&filter=name==PVDC-NUCluster
2020/03/27 10:41:39 --------------------------------------------------------------------------------
2020/03/27 10:41:39 Req header:
2020/03/27 10:41:39 X-Vcloud-Authorization: [********]
2020/03/27 10:41:39 Accept: [application/*+xml;version=29.0]
2020/03/27 10:41:39 ################################################################################
2020/03/27 10:41:39 Response caller schema.(*Resource).Apply-->vcd.resourceVcdVdcCreate-->vcd.getVcdVdcInput-->govcd.QueryProviderVdcByName-->govcd.(*VCDClient).QueryWithNotEncodedParams-->govcd.(*Client).QueryWithNotEncodedParams-->govcd.getResult-->govcd.decodeBody
2020/03/27 10:41:39 Response status 200 OK
2020/03/27 10:41:39 ################################################################################
2020/03/27 10:41:39 Response header:
2020/03/27 10:41:39 Content-Type: [application/vnd.vmware.vcloud.query.records+xml;version=29.0]
2020/03/27 10:41:39 Cache-Control: [no-store, must-revalidate]
2020/03/27 10:41:39 X-Vmware-Vcloud-Request-Id: [6774204b-3c98-45af-9530-32ad93e000fd]
2020/03/27 10:41:39 X-Vmware-Vcloud-Ceip-Id: [8d5c03a3-ac6f-4858-b11f-20fc4c6aa637]
2020/03/27 10:41:39 X-Vmware-Vcloud-Request-Execution-Time: [68]
2020/03/27 10:41:39 Vary: [Accept-Encoding, User-Agent]
2020/03/27 10:41:39 Date: [Fri, 27 Mar 2020 08:41:39 GMT]
2020/03/27 10:41:39 X-Vcloud-Authorization: [********]
2020/03/27 10:41:39 Response text: [2418] <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:common="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" xmlns:vmext="http://www.vmware.com/vcloud/extension/v1.5" xmlns:ns9="http://www.vmware.com/vcloud/versions" name="providerVdc" page="1" pageSize="25" total="1" href="https://test-host/api/query?type=providerVdc&page=1&pageSize=25&format=records&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.records+xml">
<Link rel="alternate" href="https://test-host/api/query?type=providerVdc&page=1&pageSize=25&format=references&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.references+xml"/>
<Link rel="alternate" href="https://test-host/api/query?type=providerVdc&page=1&pageSize=25&format=idrecords&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.idrecords+xml"/>
<VMWProviderVdcRecord cpuAllocationMhz="0" cpuLimitMhz="11584" cpuUsedMhz="0" isBusy="false" isDeleted="false" isEnabled="true" memoryAllocationMB="0" memoryLimitMB="17567" memoryUsedMB="0" name="PVDC-NUCluster" numberOfDatastores="4" numberOfResourcePools="1" numberOfStorageProfiles="2" numberOfVdcs="0" status="READY" storageAllocationMB="0" storageLimitMB="541404" storageUsedMB="5536" vcpuRatingMhz="1000" href="https://test-host/api/admin/providervdc/7bb0b474-8c1e-4648-a94e-967cb7d75b66" cpuOverheadMhz="0" taskStatusName="rclUpdateProviderVdcStorageProfiles" task="https://test-host/api/task/670af54b-efb7-4d3a-b0df-ab009fa8f81e" isVCEnabled="true" storageOverheadMB="0" vcName="vc1" memoryOverheadMB="0" vc="https://test-host/api/admin/extension/vimServer/fc5f8044-9bd8-4ab4-8165-ba7dac420fbc" taskStatus="success" taskDetails=" "/>
</QueryResultRecords>Just to make sure we are comparing apples to apples, could you try to pull terraform code from my branch and build it. It pulls in correct SDK andI have tested it from exactly this branch code. (have in mind version will be v2.8.0 if you use make install)
https://github.com/Didainius/terraform-provider-vcd/tree/test-pvdc-nsxt-fix
fgrehl
commented
4 years ago I tried with your branch, it still produces the same error. So the only difference is another NSX-T version? I don't know if there is any NSX-T/vCD related configuration that might affect this behavior.
# git clone https://github.com/Didainius/terraform-provider-vcd.git -b test-pvdc-nsxt-fix
# cd terraform-provider-vcd/
# make install
# terraform init
# terraform version
Terraform v0.12.24
+ provider.vcd v2.8.0
# terraform apply
[...]
vcd_org_vdc.ovdc-fgr: Creating...
Error: no provider VDC found with name PVDC-NUCluster
Query Log:
2020/03/27 12:06:23 --------------------------------------------------------------------------------
2020/03/27 12:06:23 Request caller: schema.(*Resource).Apply-->vcd.resourceVcdVdcCreate-->vcd.getVcdVdcInput-->govcd.(*VCDClient).QueryWithNotEncodedParams-->govcd.QueryProviderVdcByName
2020/03/27 12:06:23 GET https://vcloud.virten.lab/api/query?&type=providerVdc&filterEncoded=true&filter=name==PVDC-NUCluster
2020/03/27 12:06:23 --------------------------------------------------------------------------------
2020/03/27 12:06:23 Req header:
2020/03/27 12:06:23 X-Vcloud-Authorization: [********]
2020/03/27 12:06:23 Accept: [application/*+xml;version=29.0]
2020/03/27 12:06:23 ################################################################################
2020/03/27 12:06:23 Response caller schema.(*Resource).Apply-->vcd.resourceVcdVdcCreate-->vcd.getVcdVdcInput-->govcd.(*VCDClient).QueryWithNotEncodedParams-->govcd.QueryProviderVdcByName-->govcd.(*Client).QueryWithNotEncodedParams-->govcd.getResult-->govcd.decodeBody
2020/03/27 12:06:23 Response status 200 OK
2020/03/27 12:06:23 ################################################################################
2020/03/27 12:06:23 Response header:
2020/03/27 12:06:23 Vary: [Accept-Encoding, User-Agent]
2020/03/27 12:06:23 Content-Length: [1416]
2020/03/27 12:06:23 Date: [Fri, 27 Mar 2020 11:06:23 GMT]
2020/03/27 12:06:23 X-Vmware-Vcloud-Request-Id: [370c12ca-eec4-436c-802d-294931fc1f91]
2020/03/27 12:06:23 X-Vcloud-Authorization: [********]
2020/03/27 12:06:23 Content-Type: [application/vnd.vmware.vcloud.query.records+xml;version=29.0]
2020/03/27 12:06:23 X-Vmware-Vcloud-Request-Execution-Time: [38]
2020/03/27 12:06:23 Cache-Control: [no-store, must-revalidate]
2020/03/27 12:06:23 Response text: [1416] <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:common="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" xmlns:vmext="http://www.vmware.com/vcloud/extension/v1.5" xmlns:ns9="http://www.vmware.com/vcloud/versions" name="providerVdc" page="1" pageSize="25" total="0" href="https://vcloud.virten.lab/api/query?type=providerVdc&page=1&pageSize=25&format=records&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.records+xml">
<Link rel="alternate" href="https://vcloud.virten.lab/api/query?type=providerVdc&page=1&pageSize=25&format=references&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.references+xml"/>
<Link rel="alternate" href="https://vcloud.virten.lab/api/query?type=providerVdc&page=1&pageSize=25&format=idrecords&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.idrecords+xml"/>
</QueryResultRecords>
@fgrehl , Can you pull the latest commit and give another try? @vbauzysvmware has added another fix to underlying govcd SDK.
Thanks
fgrehl
commented
4 years ago @Didainius I pulled the latest version (https://github.com/Didainius/terraform-provider-vcd/commit/5c5aa1f7283ab83a0fe5f75da67129cc3b20b388) and it worked without any further modifications.
Here is the log:
2020/03/30 14:44:20 [TRACE] skipping fetch of versions because 7 are stored
2020/03/30 14:44:20 [TRACE] checking max API version against constraints '>= 31.0'
2020/03/30 14:44:20 [INFO] API version 33.0.0 satisfies constraints '>= 31.0'
2020/03/30 14:44:20 --------------------------------------------------------------------------------
2020/03/30 14:44:20 Request caller: schema.(*Provider).Apply-->schema.(*Resource).Apply-->vcd.resourceVcdVdcCreate-->vcd.getVcdVdcInput-->govcd.(*VCDClient).QueryWithNotEncodedParamsWithApiVersion-->govcd.QueryProviderVdcByName-->govcd.(*Client).QueryWithNotEncodedParamsWithApiVersion
2020/03/30 14:44:20 GET https://vcloud.virten.lab/api/query?&type=providerVdc&filterEncoded=true&filter=name==PVDC-NUCluster
2020/03/30 14:44:20 --------------------------------------------------------------------------------
2020/03/30 14:44:20 Req header:
2020/03/30 14:44:20 Accept: [application/*+xml;version=31.0]
2020/03/30 14:44:20 X-Vcloud-Authorization: [********]
2020/03/30 14:44:21 ################################################################################
2020/03/30 14:44:21 Response caller schema.(*Resource).Apply-->vcd.resourceVcdVdcCreate-->vcd.getVcdVdcInput-->govcd.(*VCDClient).QueryWithNotEncodedParamsWithApiVersion-->govcd.QueryProviderVdcByName-->govcd.(*Client).QueryWithNotEncodedParamsWithApiVersion-->govcd.getResult-->govcd.decodeBody
2020/03/30 14:44:21 Response status 200 OK
2020/03/30 14:44:21 ################################################################################
2020/03/30 14:44:21 Response header:
2020/03/30 14:44:21 X-Vmware-Vcloud-Token-Type: [Bearer]
2020/03/30 14:44:21 X-Vcloud-Authorization: [********]
2020/03/30 14:44:21 Content-Type: [application/vnd.vmware.vcloud.query.records+xml;version=31.0]
2020/03/30 14:44:21 Cache-Control: [no-store, must-revalidate]
2020/03/30 14:44:21 Content-Length: [2273]
2020/03/30 14:44:21 X-Vmware-Vcloud-Request-Id: [68094928-b63f-4710-b498-9b82c4ee7005]
2020/03/30 14:44:21 X-Vmware-Vcloud-Access-Token: [eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QGQzY2VmOTgzLTY0NTEtNGY1ZC1hMWJlLWU3N2IwYTUzNjNlNyIsImV4cCI6MTU4NTY1ODY2MCwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiJlYzExM2VkNzI5NjM0NzBhYTNhNDc0MGIxZGMyZDY5NiJ9.okO5pLmbTlLmpO31VJVfLlm2E-urtHjUzBq9f1jRKea-J1elkn93PC4ADJKv5ioHX_cxcvYgIoYAncxiYNl67goc4OZcU5abNFtkgEEx5ssqCEdzsA7JSGS9_wcB53PuXSiBvqAAn2VZjvsknXVuxGd8DCuw7s9dGMbu8io45f0QmT90HRAJXWzb1_caH7Usbzm0eipMSA8tZ-2GNhTlInQR_lcOukVDdsFRhMUFEFxdffzZZIuvWCzf2AKYwVAffnU6T9p6y4n3LYV6qrhVJDMIg8OOh0jeKOfsYcpSm7e5ZYbCfZJddU6_Rs85_GHfJ_BdAS_yJs6afNY9zVDJRQ]
2020/03/30 14:44:21 X-Vmware-Vcloud-Request-Execution-Time: [88]
2020/03/30 14:44:21 Vary: [Accept-Encoding, User-Agent]
2020/03/30 14:44:21 Date: [Mon, 30 Mar 2020 12:44:21 GMT]
2020/03/30 14:44:21 Response text: [2273] <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:common="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" xmlns:vmext="http://www.vmware.com/vcloud/extension/v1.5" xmlns:ns9="http://www.vmware.com/vcloud/versions" name="providerVdc" page="1" pageSize="25" total="1" href="https://vcloud.virten.lab/api/query?type=providerVdc&page=1&pageSize=25&format=records&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.records+xml">
<Link rel="alternate" href="https://vcloud.virten.lab/api/query?type=providerVdc&page=1&pageSize=25&format=references&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.references+xml"/>
<Link rel="alternate" href="https://vcloud.virten.lab/api/query?type=providerVdc&page=1&pageSize=25&format=idrecords&filter=name==PVDC-NUCluster&filterEncoded=true" type="application/vnd.vmware.vcloud.query.idrecords+xml"/>
<VMWProviderVdcRecord cpuAllocationMhz="0" cpuLimitMhz="25280" cpuUsedMhz="0" isBusy="false" isDeleted="false" isEnabled="true" memoryAllocationMB="0" memoryLimitMB="206094" memoryUsedMB="0" name="PVDC-NUCluster" numberOfDatastores="5" numberOfResourcePools="2" numberOfStorageProfiles="3" numberOfVdcs="2" status="READY" storageAllocationMB="20736" storageLimitMB="7993225" storageUsedMB="3007997" href="https://vcloud.virten.lab/api/admin/providervdc/b59dfe5d-3cfe-4adb-82e1-57baf868793f" cpuOverheadMhz="0" taskStatusName="rclUpdateResourcePoolSet" task="https://vcloud.virten.lab/api/task/578ee39e-151b-4705-8428-88821f65c506" isVCEnabled="true" storageOverheadMB="0" vcName="vc.virten.lab" memoryOverheadMB="0" vc="https://vcloud.virten.lab/api/admin/extension/vimServer/e13547ff-7581-42f0-8d47-d27904ecd565" taskStatus="success" taskDetails=" "/>
</QueryResultRecords>We are gradually rolling out NSX-T support and this should now be resolved without any outstanding issues. (via #550). @fgrehl , if you are interested in testing before release - it would be beneficial for us.
Versions Used:
I'm trying to create an OrgVDC. Terraform fails to find the Provider VDC.
go-vcloud-director.log:
When running the same API request manually, it does find anything either:
GET https://vcloud.virten.lab/api/query?&type=providerVdc&filter=name==PVDC-NUCluster Accept: [application/*+xml;version=29.0]
When running the same API request with Accept API Version >=31.0, i get positive results:
GET https://vcloud.virten.lab/api/query?&type=providerVdc&filter=name==PVDC-NUCluster Accept: [application/*+xml;version=31.0]