vmware / terraform-provider-vcd

Terraform VMware Cloud Director provider
https://www.terraform.io/docs/providers/vcd/
Mozilla Public License 2.0
151 stars 111 forks source link

Importing an existing UI plugin doesn't work as documented #1192

Closed JonathanWrightSE closed 10 months ago

JonathanWrightSE commented 10 months ago

I am attempting to cutomize the VCD portal by publishing an existing plugin to an organization. However the documented method for doing this doesn't work. The method is to create an empty vcd_ui_plugin resource, then to import the existing plugin into that resource which can then be manipulated. Attempting to create the empty resource fails because the resource expects to be given the "path" parameter to a zip file containing a new plugin.

Terraform Version

Terraform v1.6.6 on windows_amd64

Affected Resource(s)

Terraform Configuration Files

resource "vcd_ui_plugin" "my_plugin" { enabled = true }

Debug Output

vcd_ui_plugin.my_plugin: Creating... ╷ │ Error: could not create the UI Plugin: plugin path must not be empty │ │ with vcd_ui_plugin.my_plugin, │ on customize-portal.tf line 3, in resource "vcd_ui_plugin" "my_plugin": │ 3: resource "vcd_ui_plugin" "my_plugin" { │ ╵

Panic Output

N/A

Expected Behavior

An empty vcd_ui_plugin resource should have been created that I could then import an existing UI plugin into (as described in the vcd_ui_plugin resource documentation).

Actual Behavior

The above error occured.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

  1. terraform apply

User Access rights

N/A

Important Factoids

N/A

References

N/A

adambarreiro commented 10 months ago

Hi @JonathanWrightSE,

To perform a successful import, the UI Plugin must not be created by Terraform. It needs to exist only in the target VCD, then you would need a HCL snippet like the one you shared, but you need to import the resource before applying.

In other words, you first would prepare the empty resource, but only in code:

resource "vcd_ui_plugin" "plugin" {
  enabled     = true
}

Before doing a terraform apply, you import it from VCD:

terraform import vcd_ui_plugin.plugin VMware."Customize Portal".4.0.0

Then it should be imported correctly and the info populated in the state:

vcd_ui_plugin.plugin: Import prepared!
  Prepared vcd_ui_plugin for import
vcd_ui_plugin.plugin: Refreshing state... [id=urn:vcloud:uiPlugin:c60450ef-9536-4315-8bfc-d239b55ebf2e]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

And now it should be good to perform terraform apply or any other change.

With these steps I could not get any error, let me know if there's other operation that I didn't consider in the scenario.

JonathanWrightSE commented 10 months ago

Hi Adam

Thank you for your prompt response.

I have tried doing the import first, but mage the mistake of removing the resource from my code before doing so. Now I've done the import with the code in place it works correctly.

Many thanks for your help.

/Jonathan

carmine73 commented 1 month ago

Hi @adambarreiro, I have a strange behavior using the resource "vcd_ui_plugin", this is my code:

import {
  for_each = local.dr_plugin != null ? toset(["0"]) : [] # only if plugin is found, count cannot be used

  to = vcd_ui_plugin.dr_plugin[0]
  id = local.dr_plugin_id
}

# publish dr plugin
resource "vcd_ui_plugin" "dr_plugin" {
  count = local.dr_plugin != null ? 1 : 0 # only if plugin is found

  enabled    = true
  tenant_ids = local.dr_org_ids
}

the apply has been done at least once, so I have the resource in my state file (import is done during the first apply)

$ terraform state show 'vcd_ui_plugin.dr_plugin[0]'
# vcd_ui_plugin.dr_plugin[0]:
resource "vcd_ui_plugin" "dr_plugin" {
    description     = "VMware Cloud Director Availability™"
    enabled         = true
    id              = "urn:vcloud:uiPlugin:ec0f1e89-..."
    license         = "Copyright (C) VMware 2018-2020. All rights reserved."
    link            = "https://vcda:443"
    name            = "VCDA"
    provider_scoped = true
    status          = "ready"
    tenant_ids      = [
        "urn:vcloud:org:AAAAAAAA-...56",
        "urn:vcloud:org:AAAAAAAA-...66",
        "urn:vcloud:org:AAAAAAAA-...78",
        "urn:vcloud:org:AAAAAAAA-...60",
        "urn:vcloud:org:AAAAAAAA-...c1",
        "urn:vcloud:org:AAAAAAAA-...ec",
    ]
    tenant_scoped   = true
    vendor          = "VMware"
    version         = "4.7.1"
}

If I try a following plan, I got this message:

  # vcd_ui_plugin.dr_plugin[0] will be created
  + resource "vcd_ui_plugin" "dr_plugin" {
      + description     = (known after apply)
      + enabled         = true
      + id              = (known after apply)
      + license         = (known after apply)
      + link            = (known after apply)
      + name            = (known after apply)
      + provider_scoped = (known after apply)
      + status          = (known after apply)
      + tenant_ids      = [
          + "urn:vcloud:org:AAAAAAAA-...56",
          + "urn:vcloud:org:AAAAAAAA-...66",
          + "urn:vcloud:org:AAAAAAAA-...78",
          + "urn:vcloud:org:AAAAAAAA-...60",
          + "urn:vcloud:org:AAAAAAAA-...c1",
          + "urn:vcloud:org:AAAAAAAA-...ec",
        ]
      + tenant_scoped   = (known after apply)
      + vendor          = (known after apply)
      + version         = (known after apply)
    }

In this example the tenant_ids is unchanged (this plugin is already publish to this tenant) but if I have a change in ids I got the same issue. If I proceed with apply I will get an error:

Error: could not create the UI Plugin: plugin path must not be empty with vcd_ui_plugin.dr_plugin[0]

And if I do apply again it works. At the moment I do not run the apply, just to keep this situation. I use the same code for different plugins, if I remember well I got this issue only with this plugin.

Do I make something wrong? Is it a terraform issue?

adambarreiro commented 1 month ago

@carmine73 at first glance it looks like it tries to Create the UI Plugin instead of Updating the existing one (that's why it complains about empty .zip path). Does the terraform plan tell that theres a forced replacement, or it just tells it's going to be created?

carmine73 commented 1 month ago

no forced replacement. I'm trying something different in may code

tenant_ids = local.dr_org_ids # local var created using a data source

but I don't know if is the cause. I'll keep you in touch

carmine73 commented 1 month ago

Today I cannot reproduce the issue, I have a correct update in place. Anyway I do not use anymore the local value:

# publish dr plugin
resource "vcd_ui_plugin" "dr_plugin" {
  count = local.dr_plugin != null ? 1 : 0 # only if plugin is found

  enabled    = true
  #tenant_ids = local.dr_org_ids
  tenant_ids = (var.input.dr.required == true
    ? setunion(data.vcd_ui_plugin.dr_plugin[0].tenant_ids, [vcd_org.org.id])
    : setsubtract(data.vcd_ui_plugin.dr_plugin[0].tenant_ids, [vcd_org.org.id])
  )
}
carmine73 commented 1 month ago

Today I got the issue again.

terraform state after first apply

$ terraform state list | grep ui_plugin
data.vcd_ui_plugin.bck_plugin[0]
data.vcd_ui_plugin.dr_plugin[0]
data.vcd_ui_plugin.k8s_plugin[0]
vcd_ui_plugin.bck_plugin[0]
vcd_ui_plugin.dr_plugin[0]
vcd_ui_plugin.k8s_plugin[0]

$ terraform state show 'vcd_ui_plugin.bck_plugin[0]'
# vcd_ui_plugin.bck_plugin[0]:
resource "vcd_ui_plugin" "bck_plugin" {
    description     = "Extends vCloud Director tenant UI with self-service backup and restore options"
    . . .

terraform plan gives this output

  # vcd_ui_plugin.bck_plugin[0] will be created
  + resource "vcd_ui_plugin" "bck_plugin" {
  . . .

  # vcd_ui_plugin.dr_plugin[0] will be created
  + resource "vcd_ui_plugin" "dr_plugin" {
  . . .

  # vcd_ui_plugin.k8s_plugin[0] will be created
  + resource "vcd_ui_plugin" "k8s_plugin" {
  . . .

terraform apply

Error: could not create the UI Plugin: plugin path must not be empty

  with vcd_ui_plugin.bck_plugin[0],
  on fcp-plugins.tf line 36, in resource "vcd_ui_plugin" "bck_plugin":
  36: resource "vcd_ui_plugin" "bck_plugin" {

Error: could not create the UI Plugin: plugin path must not be empty

  with vcd_ui_plugin.dr_plugin[0],
  on fcp-plugins.tf line 61, in resource "vcd_ui_plugin" "dr_plugin":
  61: resource "vcd_ui_plugin" "dr_plugin" {

Error: could not create the UI Plugin: plugin path must not be empty

  with vcd_ui_plugin.k8s_plugin[0],
  on fcp-plugins.tf line 86, in resource "vcd_ui_plugin" "k8s_plugin":
  86: resource "vcd_ui_plugin" "k8s_plugin" {

terraform plan after apply error

  # vcd_ui_plugin.bck_plugin[0] will be imported
    resource "vcd_ui_plugin" "bck_plugin" {
    . . .

  # vcd_ui_plugin.dr_plugin[0] will be imported
    resource "vcd_ui_plugin" "dr_plugin" {
    . . .

  # vcd_ui_plugin.k8s_plugin[0] will be imported
    resource "vcd_ui_plugin" "k8s_plugin" {
    . . .

then terraform apply ok. issue on import block?

https://developer.hashicorp.com/terraform/language/import

The import block is idempotent, meaning that applying an import action and running another plan will not generate another import action as long as that resource remains in your state.

Terraform only needs to import a given resource once. Attempting to import a resource into the same address again is a harmless no-op. You can remove import blocks after completing the import or safely leave them in your configuration as a record of the resource's origin for future module maintainers

adambarreiro commented 1 month ago

Maybe the count is adding some weird side-effect in the import blocks?

carmine73 commented 1 month ago

I'll try without count, thanks (not so easy with already deployed tenants). Do you think is it a terraform core issue?

carmine73 commented 1 month ago

@adambarreiro I wasn't able to test it without count but I've seen something interesting on terraform core debug logs (I have three imported vcd_ui_plugin but only one, in this case, is marked to be created:

2024-10-09T10:41:35.703+0200 [DEBUG] expandResourceImports: skipping import address vcd_ui_plugin.dr_plugin[0] already in state
2024-10-09T10:41:35.703+0200 [DEBUG] expandResourceImports: skipping import address vcd_ui_plugin.k8s_plugin[0] already in state
2024-10-09T10:41:35.704+0200 [DEBUG] ReferenceTransformer: "vcd_ui_plugin.dr_plugin[0]" references: []
2024-10-09T10:41:35.704+0200 [DEBUG] ReferenceTransformer: "vcd_ui_plugin.k8s_plugin[0]" references: []
2024-10-09T10:41:35.704+0200 [DEBUG] expandResourceImports: skipping import address vcd_ui_plugin.bck_plugin[0] already in state
2024-10-09T10:41:35.705+0200 [DEBUG] ReferenceTransformer: "vcd_ui_plugin.bck_plugin[0]" references: []
...
2024-10-09T10:41:35.757+0200 [INFO]  provider.terraform-provider-vcd_v3.14.0: 2024/10/09 10:41:35 [DEBUG] UI Plugin no longer exists. Removing from tfstate: timestamp="2024-10-09T10:41:35.757+0200"
2024-10-09T10:41:35.757+0200 [WARN]  Provider "registry.terraform.io/vmware/vcd" produced an unexpected new value for vcd_ui_plugin.dr_plugin[0] during refresh.
      - Root object was present, but now absent

it seems something on provider side

adambarreiro commented 1 month ago

@carmine73 Thanks! Yeah, it's a bit strange...

Is it a built-in plugin from VCD?

I'll try to debug it. I see the part where it is removed from state, but this would only happen if the plugin was not found during subsequent reads, and I don't know why that happens. I'd need to debug it.

carmine73 commented 1 month ago

I have the issues on all 3 plugins, randomly. I'm pretty sure they are installed in a second moment not when vcd was created, dr_plugin is the vCDA plugin, for example

carmine73 commented 1 month ago

actually data source and the resource should point the same id, in this case it is not the same for the plugin with the issue. I'll work on it.

data.vcd_ui_plugin.dr_plugin[0]:  Read complete after 0s [id=urn:vcloud:uiPlugin:AAAAAAAA-cab1-4493-aef0-b98e9eb8cde7]
data.vcd_ui_plugin.k8s_plugin[0]: Read complete after 0s [id=urn:vcloud:uiPlugin:BBBBBBBB-e04b-4a1f-ab55-0e466a813665]
data.vcd_ui_plugin.bck_plugin[0]: Read complete after 0s [id=urn:vcloud:uiPlugin:CCCCCCCC-b491-419a-a531-d77915627e82]

vcd_ui_plugin.dr_plugin[0]:  Refreshing state... [id=urn:vcloud:uiPlugin:DDDDDDDD-5575-4ce1-b368-3a0715477412]
vcd_ui_plugin.k8s_plugin[0]: Refreshing state... [id=urn:vcloud:uiPlugin:BBBBBBBB-e04b-4a1f-ab55-0e466a813665]
vcd_ui_plugin.bck_plugin[0]: Refreshing state... [id=urn:vcloud:uiPlugin:CCCCCCCC-b491-419a-a531-d77915627e82]
carmine73 commented 1 month ago

in state file the ids are the same, further investigation by my side are required:

$ terraform state show 'data.vcd_ui_plugin.dr_plugin[0]'
# data.vcd_ui_plugin.dr_plugin[0]:
data "vcd_ui_plugin" "dr_plugin" {
    description     = "VMware Cloud Director Availability™"
    enabled         = true
    id              = "urn:vcloud:uiPlugin:DDDDDDDD-5575-4ce1-b368-3a0715477412"
    license         = "Copyright (C) VMware 2018-2020. All rights reserved."
    ... 
}

$ terraform state show 'vcd_ui_plugin.dr_plugin[0]'
# vcd_ui_plugin.dr_plugin[0]:
resource "vcd_ui_plugin" "dr_plugin" {
    description     = "VMware Cloud Director Availability™"
    enabled         = true
    id              = "urn:vcloud:uiPlugin:DDDDDDDD-5575-4ce1-b368-3a0715477412"
    license         = "Copyright (C) VMware 2018-2020. All rights reserved."
    ...
}
carmine73 commented 1 month ago

I guess I've got it. For some reason (plugin reinstallation?) the id of the plugin is changed on vcd, the correct id is the one I see in data source log (id is not used to get data):

data.vcd_ui_plugin.dr_plugin[0]:  Read complete after 0s [id=urn:vcloud:uiPlugin:AAAAAAAA-cab1-4493-aef0-b98e9eb8cde7]

the id in the state file is wrong/outdated and during plan phase is not found on vcd, so the resource must be recreated (from the terraform point of view)

edit: the id of vcda plugin has changed during a kind of reconnect from vcda to vcd