Apigee State Rendering with Computed Name Field

justin-watkinson-cherre commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

v1.2.8

Affected Resource(s)

google_apigee_organization

Terraform Configuration Files

{
    "provider": {
        "google": {
            "project": "my-project",
            "region": "us-east4"
        }
    },
    "resource": {
        "google_apigee_organization": {
            "foo": {
                "analytics_region": "us-east1",
                "authorized_network": "my-project",
                "billing_type": "EVALUATION",
                "description": "Testing",
                "display_name": "DisplayName",
                "project_id": "my-project",
                "runtime_type": "CLOUD",
                "timeouts": {
                    "create": "10m0s"
                }
            }
        }
    },
    "terraform": {
        "required_providers": {
            "google": {
                "source": "hashicorp/google",
                "version": "4.32.0"
            }
        }
    }
}

Debug Output

Panic Output

N/A

Expected Behavior

When the state file does not include the "Name' field, the ID is used to determine if the resource already exists

Actual Behavior

When the terraform refresh occurs, the statefile is rendered without a name field. A subsequent terraform plan attempts to re-create a resource which does not exist. This does not cooperate well with things life a lifecycle prevent_destroy attached to the resource.

By adding the name field to the terraform statefile, it begins to work properly. Given that name is a computed, Output only parameter

Steps to Reproduce

terraform refresh
terraform plan

Important Factoids

I am attempting to use terrajet to create a YAML representation of an Apigee Organization (and subsequent resources). When it renders the YAML, it correctly detects that the name field is not an argument, and does not add it to the YAML resources or to the rendered main.tf.json.

However, it fails to complete a plan because after doing a terraform refresh. To illustrate, terrajet renders this terraform.tfstate originally:

{
  "version": 4,
  "terraform_version": "1.2.8",
  "serial": 1,
  "lineage": "0c07d558-70e0-44c6-9710-79e41791e7e5",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "google_apigee_organization",
      "name": "foo",
      "provider": "provider[\"registry.terraform.io/hashicorp/google\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "analytics_region": "us-east1",
            "authorized_network": "foo-network",
            "billing_type": "EVALUATION",
            "ca_certificate": "",
            "description": "Testing",
            "display_name": "DisplayName",
            "id": "foo",
            "project_id": "my-project",
            "runtime_type": "CLOUD",
            "subscription_type": ""
          },
          "sensitive_attributes": [],
          "private": "<redacted>"
        }
      ]
    }
  ]
}

Upon terraform refresh the state file is then mutated to look like this:

{
  "version": 4,
  "terraform_version": "1.2.8",
  "serial": 2,
  "lineage": "0c07d558-70e0-44c6-9710-79e41791e7e5",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "google_apigee_organization",
      "name": "foo",
      "provider": "provider[\"registry.terraform.io/hashicorp/google\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "analytics_region": "",
            "authorized_network": "",
            "billing_type": "",
            "ca_certificate": "",
            "description": "",
            "display_name": "",
            "id": "foo",
            "project_id": "my-project",
            "runtime_database_encryption_key_name": "",
            "runtime_type": "",
            "subscription_type": "",
            "timeouts": null
          },
          "sensitive_attributes": [],
          "private": "<redacted>"
        }
      ]
    }
  ]
}

This state file, upon then performing a terraform plan, attempts to recreate the resource, which fails because the lifecycle prevent_destroy flag is set.

By adding a correct name field to this updated state file, I can get the terraform plan to execute without trying to re-create the resource. The apparent issue then is that the name, a computed output field, is somehow used above the id field from the state file to determine if the resource exists.

It appears that the name from the config is used to render the HTTP URL used, but may not necessarily known leading to this behavior.

Terrajet does provide facilities to pre-render or mutate the id field from known data, but it doesn't have one for name. If name is also an input attribute, then it should just work as-is, however, this particular resource does not support a name field.

References

None yet. I may link in a terrajet workaround that would hopefully let us work around this issue.

edwardmedia commented 2 years ago

@justin-watkinson-cherre by running below sample, it seems to be fine with the name field. I also ran your code above and the result seemed normal. The name was assigned with the value of the project. Can you share your debug log for your findings?

https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/apigee_organization#example-usage---apigee-organization-cloud-basic

justin-watkinson-cherre commented 2 years ago

I've attached the debug output to this comment:

debug.txt

"Bug" is probably too strong a word, but it really comes as a mismatch between the expectations of the two projects. Naturally, when using the standard terraform tooling for creation/import, it works as expected.

When terrajet renders a tfstate file, it is ignoring the name field because it's to be computed (and by description Output only) and is not available a resource input. The expectation is that the id field from the tfstate file would be used for any discovery and read operation (so far as I can tell).

The reason I looked at it from the gcp provider point of view is I believe id is the more expected value to be used for this purpose within the context of the greater terraform ecosystem. Admittedly I haven't done exhaustive research on that point, but based on where I've gotten so far this seems to be true. I've looked at other, non-GCP providers that would use resources that do not have a name input attribute and have seen id used for these same uses.

edwardmedia commented 2 years ago

@justin-watkinson-cherre I am not aware of supporting terrajets is one of the goals for this provider. As you discovered, this resource works as expected. Regarding id and name, I think generally you are right. But there are exceptions. To understand how this resource work, I also would encourage you to review below api. Does this make sense?

Naturally, when using the standard terraform tooling for creation/import, it works as expected.

https://cloud.google.com/apigee/docs/reference/apis/apigee/rest/v1/organizations

justin-watkinson-cherre commented 2 years ago

I see. The only reason I noticed is because it looks like their interface supports the id field as what is expected for most resources.

I also placed an enhancement request on their project as well to provide a workaround from when a field other than id is used somehow in the statefile to bridge this gap.

https://github.com/crossplane/terrajet/issues/299

edwardmedia commented 2 years ago

@justin-watkinson-cherre cool. Closing this then. Feel free to reopen if you want to continue the conversation

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-google