Mastercard / terraform-provider-restapi

A terraform provider to manage objects in a RESTful API
Other
782 stars 214 forks source link

The provider tries to create the resource in each terraform plan/apply execution #194

Open ivanfavi opened 1 year ago

ivanfavi commented 1 year ago

After a terraform apply the provider still try to create the resource This is the terraform code:

resource "restapi_object" "cf_ssl_recommender" {
  path          = "/client/v4/zones/${id}/settings/ssl_recommender"
  create_method = "PATCH"
  id_attribute  = var.id
  object_id     =  var.id
  data          = "{ \"enabled\": ${local.cf_ssl_recomemender_enable} }"
}

And this is the terraform output for terraform plan/apply

Terraform will perform the following actions:

  # restapi_object.cf_ssl_recommender will be created
  + resource "restapi_object" "cf_ssl_recommender" {
      + api_data        = (known after apply)
      + api_response    = (known after apply)
      + create_method   = "PATCH"
      + create_response = (known after apply)
      + data            = jsonencode(
            {
              + enabled = true
            }
        )
      + id              = (known after apply)
      + id_attribute    = XXXXXXXXX
      + object_id       = XXXXXXXXX
      + path            = "/client/v4/zones/{id}/settings/ssl_recommender"
    }

Plan: 1 to add, 0 to change, 0 to destroy.
restapi_object.cf_ssl_recommender: Creating...
restapi_object.cf_ssl_recommender: Creation complete after 1s [id=XXXXXXXXX]

We are able to see the resource with the desired information within the terraform state file after execute a terraform apply

❯ terraform state show restapi_object.cf_ssl_recommender
# restapi_object.cf_ssl_recommender:
resource "restapi_object" "cf_ssl_recommender" {
    api_data        = {
        "errors"   = jsonencode([])
        "messages" = jsonencode([])
        "result"   = "map[editable:true enabled:true id:ssl_recommender modified_on:2022-09-06T12:02:17.085Z]"
        "success"  = "true"
    }
    api_response    = jsonencode(
        {
            errors   = []
            messages = []
            result   = {
                editable    = true
                enabled     = true
                id          = "ssl_recommender"
                modified_on = "2022-09-06T12:02:17.085Z"
            }
            success  = true
        }
    )
    create_method   = "PATCH"
    create_response = jsonencode(
        {
            errors   = []
            messages = []
            result   = {
                editable    = true
                enabled     = true
                id          = "ssl_recommender"
                modified_on = "2022-09-06T12:02:17.085Z"
            }
            success  = true
        }
    )
    data            = jsonencode(
        {
            enabled = true
        }
    )
    id              = "XXXXXXXX"
    id_attribute    = "XXXXXXXX"
    object_id       = "XXXXXXXX"
    path            = "/client/v4/zones/XXXXXXXX/settings/ssl_recommender"
}

Is this the right expected behavior of this resource? We would like to create it once time only

briananstett commented 1 year ago

I'm running into the same issue. Given that the provider supports "CRUD" I was expecting the create (POST) endpoint to be hit on the initial terraform apply, the READ (GET) endpoint to be hit on subsequent terraform apply commands, the update (PUT) endpoint to update the object if there were any changes, and finally the delete (DELETE) endpoint on terraform destroy

I really hope we're just doing something wrong :crossed_fingers:

briananstett commented 1 year ago

@ivanfavi it appears the provider does support updating an existing resource and will not create a new if no changes are detected.

My issues was I had a trailing / in the path which was cuasing the GET request to fail.

2022-10-03T19:35:44.229-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go: method='GET', path='/spreadsheet/1CYiQ3SHQJcY6k_rwiuXZSLNZAD4axfjoQHFpsvOyHDo/sheet/0/row//a84c6a12-eb40-4af1-bd86-acf55cc35a5f', full uri (derived)='http://127.0.0.1:3000/spreadsheet/1CYiQ3SHQJcY6k_rwiuXZSLNZAD4axfjoQHFpsvOyHDo/sheet/0/row//a84c6a12-eb40-4af1-bd86-acf55cc35a5f', data='': timestamp=2022-10-03T19:35:44.229-0400
2022-10-03T19:35:44.229-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go: Sending HTTP request to http://127.0.0.1:3000/spreadsheet/1CYiQ3SHQJcY6k_rwiuXZSLNZAD4axfjoQHFpsvOyHDo/sheet/0/row//a84c6a12-eb40-4af1-bd86-acf55cc35a5f...: timestamp=2022-10-03T19:35:44.229-0400
2022-10-03T19:35:44.229-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go: Request headers:: timestamp=2022-10-03T19:35:44.229-0400
2022-10-03T19:35:44.229-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go: BODY:: timestamp=2022-10-03T19:35:44.229-0400
2022-10-03T19:35:44.229-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 <none>: timestamp=2022-10-03T19:35:44.229-0400
2022-10-03T19:35:44.229-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 Waiting for rate limit availability: timestamp=2022-10-03T19:35:44.229-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go: Response code: 404: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go: Response headers:: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go:   Date: Mon, 03 Oct 2022 23:35:44 GMT: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go:   Connection: keep-alive: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go:   Keep-Alive: timeout=5: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go:   X-Powered-By: Express: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go:   Content-Security-Policy: default-src 'none': timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go:   X-Content-Type-Options: nosniff: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go:   Content-Type: text/html; charset=utf-8: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go:   Content-Length: 245: timestamp=2022-10-03T19:35:44.230-0400
2022-10-03T19:35:44.230-0400 [INFO]  provider.terraform-provider-restapi_v1.17.0: 2022/10/03 19:35:44 api_client.go: BODY:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /spreadsheet/1CYiQ3SHQJcY6k_rwiuXZSLNZAD4axfjoQHFpsvOyHDo/sheet/0/row//a84c6a12-eb40-4af1-bd86-acf55cc35a5f</pre>
</body>
</html>: timestamp=2022-10-03T19:35:44.230-0400

Try running your terraform apply/plan with info logs available to see what the REST API is returning

TF_LOG=info terraform apply

I'm not sure if you're having the same issue as me but I don't think this is an issue with the provider.

ivanfavi commented 1 year ago

Adding a trailing / I get this error message: unexpected response code '404': 404 page not found It seems the API endpoint doesn't support this trailing / at the end: https://api.cloudflare.com/#zone-settings-change-ssl/tls-recommender-enrollment

As a workaround I added some logic through terraform

data "http" "cf_ssl_recommender_status" {
  url = "https://api.cloudflare.com/client/v4/zones/${zone_id}/settings/ssl_recommender"
  request_headers = {
    Content-Type  = "application/json"
  }
}

locals {
  ssl_recomemender_enable       = true
  ssl_recomemender_api_response = jsondecode(data.http.cf_ssl_recommender_status.response_body)
  ssl_recommender_status        = lookup(lookup(local.ssl_recomemender_api_response, "result"), "enabled")
}

resource "restapi_object" "cf_ssl_recommender" {
  count         = local.ssl_recomemender_enable != local.ssl_recommender_status ? 1 : 0
  path          = "/client/v4/zones/${zone_id}/settings/ssl_recommender/"
  create_method = "PATCH"
  id_attribute  = id
  object_id     = id
  data          = "{ \"enabled\": ${local.ssl_recomemender_enable} }"

  depends_on = [
    local.ssl_recomemender_api_response
  ]

}

Maybe the issue belongs to the API that I'm working on and it is not related to the terraform provider restapi.

Feel free to close this issue if you consider

tkriviradev commented 11 months ago

I am also experiencing the same issue. Terraform state is updated however, the provider has rouge behavior.

It should be considered a bug.

ikegentz commented 10 months ago

I thought I was having this issue as well, however I needed to set the id_attribute in the provider config. When I set TF_LOG=debug I was able to see the provider trying to do a GET on my resources I'd just created, however the provider I was using uses name as the id instead of the actual id returned by a POST, so this GET would always fail, causing it to try and re-create every time.

Double check the API you're using, for how you'd look up a single item, and make sure you've got id_attribute set correctly in your provider config

gbish commented 7 months ago

I'm having a similar issue with the Shopify API.

I'm attempting to register webhooks, and for the most part the existing Shopify providers on the registry work fine, but one of the webhooks I'm trying to register requires a specific API version (2023-07 or later), which said providers don't support.

So I'm currently able to register the webhook with the following config:

provider "restapi" {
  uri                  = "https://${var.shop}.myshopify.com"
  write_returns_object = true
  debug                = true
  id_attribute         = "webhook/id"

  headers = {
    "Accept"                 = "application/json",
    "X-Shopify-Access-Token" = data.aws_ssm_parameter.access_token.value
  }
}
resource "restapi_object" "customers_email_marketing_consent_update" {
  path      = "/admin/api/${var.shopify_api_version}/webhooks.json"
  read_path = "/admin/api/${var.shopify_api_version}/webhooks/{id}.json"
  object_id = "webhook/id"

  data = jsonencode({
    webhook = {
      topic   = "customers_email_marketing_consent/update"
      address = data.aws_cloudwatch_event_source.shopify.arn
      format  = "json"
    }
  })
}

However, when making subsequent plan applications, it always attempts to recreate those webhooks, which results in a 422: {"errors":{"address":["for this topic has already been taken"]}}

After enabling the TF_LOG env var, I can only see GET requests for the original provider, I can't see this provider attempting to ascertain the state of these webhook resources.

The only output related to restapi is the following:

2023-12-06T17:50:35.523Z [INFO]  provider.terraform-provider-restapi_v1.18.2: configuring server automatic mTLS: timestamp=2023-12-06T17:50:35.523Z
2023-12-06T17:50:35.562Z [INFO]  provider: configuring client automatic mTLS
2023-12-06T17:50:35.653Z [WARN]  ValidateProviderConfig from "module.shopify_events_gb.provider[\"registry.terraform.io/mastercard/restapi\"]" changed the config value, but that value is unused
2023-12-06T17:50:35.655Z [INFO]  provider.terraform-provider-restapi_v1.18.2: 2023/12/06 17:50:35 api_client.go: Constructing debug api_client: timestamp=2023-12-06T17:50:35.655Z
2023-12-06T17:50:35.655Z [INFO]  provider.terraform-provider-restapi_v1.18.2: 2023/12/06 17:50:35 limit: 179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.000000 bucket: -9223372036854775808: timestamp=2023-12-06T17:50:35.655Z
2023-12-06T17:50:35.655Z [INFO]  provider.terraform-provider-restapi_v1.18.2: 2023/12/06 17:50:35 api_client.go: Constructed client:
uri: https://my-shop.myshopify.com
insecure: false
username:
password:
id_attribute: webhook/id
write_returns_object: true
create_returns_object: false
headers:
Accept: application/json
X-Shopify-Access-Token: ***: timestamp=2023-12-06T17:50:35.655Z