search

scastria / terraform-provider-apigee

Mozilla Public License 2.0
21 stars 18 forks source link

Apigee ng #5

Closed davidmitchell2019 closed 3 years ago

davidmitchell2019 commented 3 years ago

Hello, I am trying to use the provider to deploy an API proxy to Apigee ng but I keep getting issues with the authentication via an access token.

Error: Status 401: Message: Unauthorized:

Unauthorized

Error 401

I am creating an access token via my logged in account that has access to the apigee ng instance:

gcloud auth application-default print-access-token

scastria commented 3 years ago

What is NG?

maybe your token is not authorized to perform the specific operation the Apigee provider is trying to execute? what was the provider trying to do: create something, update something, delete something, etc?

davidmitchell2019 commented 3 years ago

Apigee ng is the latest apigee version that is google managed but can be peered into a VPC. https://docs.apigee.com/api-platform/faq/compare-apigeegcp-edge#api-summary-changes Just tested with a service account, created a token and then tried to use the token, my service account has apigee org admin role im trying to deploy an api proxy

davidmitchell2019 commented 3 years ago

I am using this for server variable: server = "apigee.google.com"

davidmitchell2019 commented 3 years ago

if i use server: server = "apigee.googleapis.com", I get this error:

Error: Status 404: Message: Not Found: <!DOCTYPE html>

Error 404 (Not Found)!!1

404. That’s an error.

The requested URL /v1/o/dev1-org/apis?action=import&name=Terraform-test was not found on this server. That’s all we know.

davidmitchell2019 commented 3 years ago

To curl the api i use this address:

https://apigee.googleapis.com/v1/organizations/$ORG/

This works with the same token

davidmitchell2019 commented 3 years ago

Here is the latest API reference.. https://cloud.google.com/apigee/docs/reference

scastria commented 3 years ago

Let me add some more debugging log messages in the provider so we can see the exact API call that is failing.

davidmitchell2019 commented 3 years ago

Thanks Scastria, it might be something to do with the way you append the url:

/v1/o/org/apis?action=import&name=Terraform-test (you append o for organization)

The curl API address has organizations in it:

https://apigee.googleapis.com/v1/organizations/$ORG/

scastria commented 3 years ago

Yes, in the past, o was valid as a substitute for organization and e was valid as a substitute for environment. I can easily change it no problem. However, just to test out my new debugging statements. Can you try out v0.1.23 after it is done building and set your TF_LOG env to DEBUG and try your test again. If you search the output for "Apigee Management API:", it should show every REST call being made so you can see the exact one that causes 401 or 404.

Then you can try the exact same REST call with CURL or PostMan to see if you can reproduce.

scastria commented 3 years ago

By the way, a previous Issue that I closed here in github had to do with public cloud versus private cloud version of Apigee. The Management API works differently in some cases depending on whether the server is public or private. Therefore, I use the server value to determine if I should use public or private compatible rest calls. this means I am comparing the server value to "api.enterprise.apigee.com". Are you saying that apigee.googleapis.com is another server that should be considered public cloud?

davidmitchell2019 commented 3 years ago

Yep, that would be correct, when i log into the portal this is the url: https://apigee.google.com/ when i use the API I use: https://apigee.googleapis.com/v1/organizations/$ORG/

I will test the above

scastria commented 3 years ago

Interesting. It seems Google keeps changing things. The documentation for the management API just recently changed. If you look here: https://apidocs.apigee.com/docs/api-proxies/1/overview you can see reference to the old server name. So this means there are 2 places for management API documentation and each references a different server name.

davidmitchell2019 commented 3 years ago

venv) ddmc@UKPC010969 apigeeTerraform % export TF_LOG=DEBUG (venv) ddmc@UKPC010969 apigeeTerraform % terraform apply 2021/01/04 19:55:28 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility. Use TF_LOG=TRACE to see Terraform's internal logs.

2021/01/04 19:55:28 [INFO] Terraform version: 0.13.5
2021/01/04 19:55:28 [INFO] Go runtime version: go1.14.12 2021/01/04 19:55:28 [INFO] CLI args: []string{"/usr/local/bin/terraform", "apply"} 2021/01/04 19:55:28 [DEBUG] Attempting to open CLI config file: /Users/ddmc/.terraformrc 2021/01/04 19:55:28 [DEBUG] File doesn't exist, but doesn't need to. Ignoring. 2021/01/04 19:55:28 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins 2021/01/04 19:55:28 [DEBUG] ignoring non-existing provider search directory /Users/ddmc/.terraform.d/plugins 2021/01/04 19:55:28 [DEBUG] ignoring non-existing provider search directory /Users/ddmc/Library/Application Support/io.terraform/plugins 2021/01/04 19:55:28 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins 2021/01/04 19:55:28 [INFO] CLI command args: []string{"apply"} 2021/01/04 19:55:28 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility. Use TF_LOG=TRACE to see Terraform's internal logs.

2021/01/04 19:55:28 [DEBUG] New state was assigned lineage "c891a92a-bf3d-87fa-68c5-48cedd78a2fd" 2021/01/04 19:55:28 [DEBUG] checking for provisioner in "." 2021/01/04 19:55:28 [DEBUG] checking for provisioner in "/usr/local/bin" 2021/01/04 19:55:28 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory 2021/01/04 19:55:28 [INFO] backend/local: starting Apply operation 2021-01-04T19:55:28.815Z [INFO] plugin: configuring client automatic mTLS 2021-01-04T19:55:28.839Z [DEBUG] plugin: starting plugin: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 args=[.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23] 2021-01-04T19:55:28.841Z [DEBUG] plugin: plugin started: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40009 2021-01-04T19:55:28.841Z [DEBUG] plugin: waiting for RPC address: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 2021-01-04T19:55:28.848Z [INFO] plugin.terraform-provider-apigee_v0.1.23: configuring server automatic mTLS: timestamp=2021-01-04T19:55:28.848Z 2021-01-04T19:55:28.871Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: plugin address: address=/var/folders/yn/4typhb6131b3h43g07vx57p1tmlbtt/T/plugin424920876 network=unix timestamp=2021-01-04T19:55:28.871Z 2021-01-04T19:55:28.871Z [DEBUG] plugin: using plugin: version=5 2021-01-04T19:55:28.918Z [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2021-01-04T19:55:28.919Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40009 2021-01-04T19:55:28.919Z [DEBUG] plugin: plugin exited 2021/01/04 19:55:28 [INFO] terraform: building graph: GraphTypeValidate 2021/01/04 19:55:28 [DEBUG] ProviderTransformer: "apigee_proxy.example" (terraform.NodeValidatableResource) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:28 [DEBUG] ProviderTransformer: "apigee_proxy_deployment.example" (terraform.NodeValidatableResource) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:28 [DEBUG] ReferenceTransformer: "apigee_proxy.example" references: [] 2021/01/04 19:55:28 [DEBUG] ReferenceTransformer: "apigee_proxy_deployment.example" references: [apigee_proxy.example] 2021/01/04 19:55:28 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/scastria/apigee\"]" references: [] 2021/01/04 19:55:28 [DEBUG] Starting graph walk: walkValidate 2021-01-04T19:55:28.920Z [INFO] plugin: configuring client automatic mTLS 2021-01-04T19:55:28.946Z [DEBUG] plugin: starting plugin: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 args=[.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23] 2021-01-04T19:55:28.949Z [DEBUG] plugin: plugin started: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40010 2021-01-04T19:55:28.950Z [DEBUG] plugin: waiting for RPC address: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 2021-01-04T19:55:28.961Z [INFO] plugin.terraform-provider-apigee_v0.1.23: configuring server automatic mTLS: timestamp=2021-01-04T19:55:28.960Z 2021-01-04T19:55:28.994Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: plugin address: address=/var/folders/yn/4typhb6131b3h43g07vx57p1tmlbtt/T/plugin483600849 network=unix timestamp=2021-01-04T19:55:28.994Z 2021-01-04T19:55:28.994Z [DEBUG] plugin: using plugin: version=5 2021-01-04T19:55:29.063Z [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2021-01-04T19:55:29.064Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40010 2021-01-04T19:55:29.065Z [DEBUG] plugin: plugin exited 2021/01/04 19:55:29 [INFO] backend/local: apply calling Refresh 2021/01/04 19:55:29 [INFO] terraform: building graph: GraphTypeRefresh 2021/01/04 19:55:29 [DEBUG] ProviderTransformer: "apigee_proxy.example (expand)" (terraform.nodeExpandRefreshableManagedResource) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:29 [DEBUG] ProviderTransformer: "apigee_proxy_deployment.example (expand)" (terraform.nodeExpandRefreshableManagedResource) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:29 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/scastria/apigee\"]" references: [] 2021/01/04 19:55:29 [DEBUG] ReferenceTransformer: "apigee_proxy.example (expand)" references: [] 2021/01/04 19:55:29 [DEBUG] ReferenceTransformer: "apigee_proxy_deployment.example (expand)" references: [apigee_proxy.example (expand)] 2021/01/04 19:55:29 [DEBUG] Starting graph walk: walkRefresh 2021-01-04T19:55:29.066Z [INFO] plugin: configuring client automatic mTLS 2021-01-04T19:55:29.090Z [DEBUG] plugin: starting plugin: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 args=[.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23] 2021-01-04T19:55:29.093Z [DEBUG] plugin: plugin started: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40011 2021-01-04T19:55:29.093Z [DEBUG] plugin: waiting for RPC address: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 2021-01-04T19:55:29.101Z [INFO] plugin.terraform-provider-apigee_v0.1.23: configuring server automatic mTLS: timestamp=2021-01-04T19:55:29.100Z 2021-01-04T19:55:29.126Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: plugin address: address=/var/folders/yn/4typhb6131b3h43g07vx57p1tmlbtt/T/plugin821487302 network=unix timestamp=2021-01-04T19:55:29.126Z 2021-01-04T19:55:29.126Z [DEBUG] plugin: using plugin: version=5 2021/01/04 19:55:29 [DEBUG] Resource state not found for node "apigee_proxy.example", instance apigee_proxy.example 2021/01/04 19:55:29 [DEBUG] Resource instance state not found for node "apigee_proxy_deployment.example", instance apigee_proxy_deployment.example 2021-01-04T19:55:29.178Z [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2021-01-04T19:55:29.179Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40011 2021-01-04T19:55:29.179Z [DEBUG] plugin: plugin exited 2021/01/04 19:55:29 [INFO] backend/local: apply calling Plan 2021/01/04 19:55:29 [INFO] terraform: building graph: GraphTypePlan 2021/01/04 19:55:29 [DEBUG] ProviderTransformer: "apigee_proxy.example (expand)" (terraform.nodeExpandPlannableResource) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:29 [DEBUG] ProviderTransformer: "apigee_proxy_deployment.example (expand)" (terraform.nodeExpandPlannableResource) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:29 [DEBUG] ReferenceTransformer: "apigee_proxy.example (expand)" references: [] 2021/01/04 19:55:29 [DEBUG] ReferenceTransformer: "apigee_proxy_deployment.example (expand)" references: [apigee_proxy.example (expand)] 2021/01/04 19:55:29 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/scastria/apigee\"]" references: [] 2021/01/04 19:55:29 [DEBUG] Starting graph walk: walkPlan 2021-01-04T19:55:29.180Z [INFO] plugin: configuring client automatic mTLS 2021-01-04T19:55:29.202Z [DEBUG] plugin: starting plugin: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 args=[.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23] 2021-01-04T19:55:29.204Z [DEBUG] plugin: plugin started: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40012 2021-01-04T19:55:29.204Z [DEBUG] plugin: waiting for RPC address: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 2021-01-04T19:55:29.211Z [INFO] plugin.terraform-provider-apigee_v0.1.23: configuring server automatic mTLS: timestamp=2021-01-04T19:55:29.211Z 2021-01-04T19:55:29.234Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: plugin address: address=/var/folders/yn/4typhb6131b3h43g07vx57p1tmlbtt/T/plugin371180011 network=unix timestamp=2021-01-04T19:55:29.233Z 2021-01-04T19:55:29.234Z [DEBUG] plugin: using plugin: version=5 2021/01/04 19:55:29 [DEBUG] Resource instance state not found for node "apigee_proxy.example", instance apigee_proxy.example 2021/01/04 19:55:29 [DEBUG] ReferenceTransformer: "apigee_proxy.example" references: [] 2021/01/04 19:55:29 [DEBUG] Resource instance state not found for node "apigee_proxy_deployment.example", instance apigee_proxy_deployment.example 2021/01/04 19:55:29 [DEBUG] ReferenceTransformer: "apigee_proxy_deployment.example" references: [] 2021-01-04T19:55:29.283Z [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2021-01-04T19:55:29.284Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40012 2021-01-04T19:55:29.284Z [DEBUG] plugin: plugin exited

An execution plan has been generated and is shown below. 2021/01/04 19:55:29 [DEBUG] command: asking for input: "Do you want to perform these actions?" Resource actions are indicated with the following symbols:

Terraform will perform the following actions:

apigee_proxy.example will be created

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve.

Enter a value: yes

2021/01/04 19:55:30 [INFO] backend/local: apply calling Apply 2021/01/04 19:55:30 [INFO] terraform: building graph: GraphTypeApply 2021/01/04 19:55:30 [DEBUG] Resource state not found for node "apigee_proxy.example", instance apigee_proxy.example 2021/01/04 19:55:30 [DEBUG] Resource state not found for node "apigee_proxy_deployment.example", instance apigee_proxy_deployment.example 2021/01/04 19:55:30 [DEBUG] ProviderTransformer: "apigee_proxy_deployment.example" (terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:30 [DEBUG] ProviderTransformer: "apigee_proxy.example (expand)" (terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:30 [DEBUG] ProviderTransformer: "apigee_proxy_deployment.example (expand)" (terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:30 [DEBUG] ProviderTransformer: "apigee_proxy.example" (terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/scastria/apigee"] 2021/01/04 19:55:30 [DEBUG] ReferenceTransformer: "apigee_proxy.example" references: [] 2021/01/04 19:55:30 [DEBUG] ReferenceTransformer: "apigee_proxy_deployment.example" references: [apigee_proxy.example apigee_proxy.example apigee_proxy.example (expand)] 2021/01/04 19:55:30 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/scastria/apigee\"]" references: [] 2021/01/04 19:55:30 [DEBUG] ReferenceTransformer: "apigee_proxy.example (expand)" references: [] 2021/01/04 19:55:30 [DEBUG] ReferenceTransformer: "apigee_proxy_deployment.example (expand)" references: [] 2021/01/04 19:55:30 [DEBUG] Starting graph walk: walkApply 2021-01-04T19:55:30.614Z [INFO] plugin: configuring client automatic mTLS 2021-01-04T19:55:30.637Z [DEBUG] plugin: starting plugin: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 args=[.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23] 2021-01-04T19:55:30.640Z [DEBUG] plugin: plugin started: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40018 2021-01-04T19:55:30.640Z [DEBUG] plugin: waiting for RPC address: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 2021-01-04T19:55:30.646Z [INFO] plugin.terraform-provider-apigee_v0.1.23: configuring server automatic mTLS: timestamp=2021-01-04T19:55:30.646Z 2021-01-04T19:55:30.669Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: plugin address: address=/var/folders/yn/4typhb6131b3h43g07vx57p1tmlbtt/T/plugin047420937 network=unix timestamp=2021-01-04T19:55:30.669Z 2021-01-04T19:55:30.669Z [DEBUG] plugin: using plugin: version=5 apigee_proxy.example: Creating... 2021/01/04 19:55:30 [DEBUG] EvalApply: ProviderMeta config value set 2021/01/04 19:55:30 [DEBUG] apigee_proxy.example: applying the planned Create change 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: 2021/01/04 19:55:30 Apigee Management API: POST /v1/o/org/-18fd/apis?action=import&name=test HTTP/1.1 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: Host: apigee.googleapis.com:443 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: Content-Type: multipart/form-data; boundary=68a708c3b7a71cf771d6d3af76c30011c681076ff26f1a6eb64edc13459c 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: --68a708c3b7a71cf771d6d3af76c30011c681076ff26f1a6eb64edc13459c 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: Content-Disposition: form-data; name="bundle"; filename="proxy_bundle.zip" 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: Content-Type: application/octet-stream 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: PK proxy_bundle/PK5:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: 7]$R �Aproxy_bundle/PK;+.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: 7]$R 2021-01-04T19:55:30.719Z [DEBUG] plugin.terraform-provider-apigee_v0.1.23: --68a708c3b7a71cf771d6d3af76c30011c681076ff26f1a6eb64edc13459c-- 2021/01/04 19:55:31 [DEBUG] apigee_proxy.example: apply errored, but we're indicating that via the Error pointer rather than returning it: Status 404: Message: Not Found: <!DOCTYPE html>

Error 404 (Not Found)!!1

404. That’s an error.

The requested URL /v1/o/dev1-org/apis?action=import&name=test was not found on this server. That’s all we know. 2021/01/04 19:55:31 [ERROR] eval: *terraform.EvalApplyPost, err: Status 404: Message: Not Found: Error 404 (Not Found)!!1

404. That’s an error.

The requested URL /v1/o/dev1-org/apis?action=import&name=test was not found on this server. That’s all we know. 2021/01/04 19:55:31 [ERROR] eval: *terraform.EvalSequence, err: Status 404: Message: Not Found: name=viewport content="initial-scale=1, minimum-scale=1, width=device-width"> Error 404 (Not Found)!!1

404. That’s an error.

The requested URL /v1/o/dev1-org/apis?action=import&name=test was not found on this server. That’s all we know. Error: Status 404: Message: Not Found: Error 404 (Not Found)!!1

404. That’s an error.

The requested URL /v1/o/dev1-org/apis?action=import&name=test was not found on this server. That’s all we know. 2021-01-04T19:55:31.443Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64/terraform-provider-apigee_v0.1.23 pid=40018 2021-01-04T19:55:31.443Z [DEBUG] plugin: plugin exited

davidmitchell2019 commented 3 years ago

Yeh there is now a few versions of Apigee, (Edge, hybrid and NG)

scastria commented 3 years ago

This new Google Cloud version (NG?) seems to have changed a lot compared to the older version! this could require a massive change to the provider. Let me investigate further... I will start by creating my own account so I can test myself.

davidmitchell2019 commented 3 years ago

Might be a little problem as this ones not quite released at the moment, you might be able to get an early access trial from your Google rep..

scastria commented 3 years ago

I was able to get an eval organization provisioned for the Google Cloud Apigee version. The API might actually be pretty close. One big difference I noticed is that the Google Cloud version doesn't have Virtual Host concept. However, every API I checked does seem to match except I need to change o to organizations and e to environments (as you suggested).

Please try v0.1.24 (after it finished building) and see if you can get farther in your testing.

davidmitchell2019 commented 3 years ago

getting closer

davidmitchell2019 commented 3 years ago

2021/01/04 22:14:17 [DEBUG] apigee_proxy.example: apply errored, but we're indicating that via the Error pointer rather than returning it: Status 400: Message: Bad Request: { "error": { "code": 400, "message": "bundle contains errors", "status": "INVALID_ARGUMENT", "details": [ { "@type": "type.googleapis.com/edge.configstore.bundle.BadBundle", "violations": [ { "description": "The proxy must contain a ProxyEndpoint" } ] }, { "@type": "type.googleapis.com/google.rpc.RequestInfo", "requestId": "12210902133209329457" } ] } } 2021/01/04 22:14:17 [ERROR] eval: terraform.EvalApplyPost, err: Status 400: Message: Bad Request: { "error": { "code": 400, "message": "bundle contains errors", "status": "INVALID_ARGUMENT", "details": [ { "@type": "type.googleapis.com/edge.configstore.bundle.BadBundle", "violations": [ { "description": "The proxy must contain a ProxyEndpoint" } ] }, { "@type": "type.googleapis.com/google.rpc.RequestInfo", "requestId": "12210902133209329457" } ] } } 2021/01/04 22:14:17 [ERROR] eval: terraform.EvalSequence, err: Status 400: Message: Bad Request: { "error": { "code": 400, "message": "bundle contains errors", "status": "INVALID_ARGUMENT", "details": [ { "@type": "type.googleapis.com/edge.configstore.bundle.BadBundle", "violations": [ { "description": "The proxy must contain a ProxyEndpoint" } ] }, { "@type": "type.googleapis.com/google.rpc.RequestInfo", "requestId": "12210902133209329457" } ] } }

Error: Status 400: Message: Bad Request: { "error": { "code": 400, "message": "bundle contains errors", "status": "INVALID_ARGUMENT", "details": [ { "@type": "type.googleapis.com/edge.configstore.bundle.BadBundle", "violations": [ { "description": "The proxy must contain a ProxyEndpoint" } ] }, { "@type": "type.googleapis.com/google.rpc.RequestInfo", "requestId": "12210902133209329457" } ] } }

2021-01-04T22:14:17.790Z [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2021-01-04T22:14:17.792Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/scastria/apigee/0.1.24/darwin_amd64/terraform-provider-apigee_v0.1.24 pid=41030 2021-01-04T22:14:17.792Z [DEBUG] plugin: plugin exited

davidmitchell2019 commented 3 years ago

i am using a no route in this api proxy, and just running some key value map operations

scastria commented 3 years ago

I just tested and it works for me. Here is my bundle zip which you can verify works for you. If it works, then I would say it is an error in your zip or perhaps an error in the Google Cloud Apigee bundle validation.

My tf file:

resource "apigee_proxy" "MyProxy" { name = "hello-world" bundle = "proxies/hello-world/hello-world.zip" bundle_hash = filebase64sha256("proxies/hello-world/hello-world.zip") }

hello-world.zip

scastria commented 3 years ago

If there is nothing sensitive inside your bundle zip, you can also send it to me and I can try to import it.

davidmitchell2019 commented 3 years ago

it applies, does yours destroy or can you plan? as im getting terraform crashed now, here is the crash.log file contents

davidmitchell2019 commented 3 years ago

brb

scastria commented 3 years ago

In order for me to destroy, I need to import the proxy_deployment first so tf will know to undeploy first. when I try to do that, I get a crash. I am investigating...

davidmitchell2019 commented 3 years ago

2021/01/04 22:26:07 [INFO] Terraform version: 0.13.5
2021/01/04 22:26:07 [INFO] Go runtime version: go1.14.12 2021/01/04 22:26:07 [INFO] CLI args: []string{"/usr/local/bin/terraform", "destroy"} 2021/01/04 22:26:07 [DEBUG] Attempting to open CLI config file: /Users/ddmc/.terraformrc 2021/01/04 22:26:07 [DEBUG] File doesn't exist, but doesn't need to. Ignoring. 2021/01/04 22:26:07 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins 2021/01/04 22:26:07 [DEBUG] ignoring non-existing provider search directory /Users/ddmc/.terraform.d/plugins 2021/01/04 22:26:07 [DEBUG] ignoring non-existing provider search directory /Users/ddmc/Library/Application Support/io.terraform/plugins 2021/01/04 22:26:07 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins 2021/01/04 22:26:07 [INFO] CLI command args: []string{"destroy"} 2021/01/04 22:26:07 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility. Use TF_LOG=TRACE to see Terraform's internal logs.

2021/01/04 22:26:07 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config 2021/01/04 22:26:07 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory 2021/01/04 22:26:07 [DEBUG] New state was assigned lineage "dbaf085c-e9fc-661e-965a-2ee34d5f3354" 2021/01/04 22:26:07 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend) 2021/01/04 22:26:07 [TRACE] Meta.Backend: instantiated backend of type 2021/01/04 22:26:07 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins 2021/01/04 22:26:07 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins 2021/01/04 22:26:07 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/scastria/apigee v0.1.22 for darwin_amd64 at .terraform/plugins/registry.terraform.io/scastria/apigee/0.1.22/darwin_amd64 2021/01/04 22:26:07 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/scastria/apigee v0.1.23 for darwin_amd64 at .terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64 2021/01/04 22:26:07 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/scastria/apigee v0.1.24 for darwin_amd64 at .terraform/plugins/registry.terraform.io/scastria/apigee/0.1.24/darwin_amd64 2021/01/04 22:26:07 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/scastria/apigee/0.1.22/darwin_amd64 as a candidate package for registry.terraform.io/scastria/apigee 0.1.22 2021/01/04 22:26:07 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/scastria/apigee/0.1.23/darwin_amd64 as a candidate package for registry.terraform.io/scastria/apigee 0.1.23 2021/01/04 22:26:07 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/scastria/apigee/0.1.24/darwin_amd64 as a candidate package for registry.terraform.io/scastria/apigee 0.1.24 2021/01/04 22:26:07 [DEBUG] checking for provisioner in "." 2021/01/04 22:26:07 [DEBUG] checking for provisioner in "/usr/local/bin" 2021/01/04 22:26:07 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory 2021/01/04 22:26:07 [TRACE] Meta.Backend: backend does not support operations, so wrapping it in a local backend 2021/01/04 22:26:07 [INFO] backend/local: starting Apply operation 2021/01/04 22:26:07 [TRACE] backend/local: requesting state manager for workspace "default" 2021/01/04 22:26:07 [TRACE] backend/local: state manager for workspace "default" will:

davidmitchell2019 commented 3 years ago

Enjoy, it is late in the UK so i need to close down, let me know how you get on and thanks for all your help

scastria commented 3 years ago

Sure, np. My current crash is due to GCP version returning different output than old version. This is what I was afraid of. I have to now test each REST call I make and compare the JSON output from old and new and support both. What a pain. Thanks Google!!

scastria commented 3 years ago

With v0.1.25, I was able to import, create, and delete a proxy deployment. I scanned your crash output and it matched my crash. Obviously, I have not tested all resource types so I am sure there will be other bugs at this time.

davidmitchell2019 commented 3 years ago

Awesome, in sure i will find them, can you share how you imported a proxy?

scastria commented 3 years ago

Well, because there is no way to import a bundle value and bundle_hash value from the server since these are client side concepts, importing a proxy is a bit convoluted. Here is an example procedure:

  1. Create a tf file with both proxy and proxy_deployment resources like this: 
    resource "apigee_proxy" "MyProxy" {
name = "hello-world"
bundle = "proxies/hello-world/hello-world.zip"
bundle_hash = filebase64sha256("proxies/hello-world/hello-world.zip")
}
resource "apigee_proxy_deployment" "MyProxyDeployment" {
proxy_name = apigee_proxy.MyProxy.name
environment_name = "eval"
revision = apigee_proxy.MyProxy.revision
}
  2. Import the proxy: terraform import apigee_proxy.MyProxy hello-world
  3. Import the proxy deployment: terraform import apigee_proxy_deployment.MyProxyDeployment eval:hello-world
  4. Check the import with a plan: terraform plan

  5. Notice the proxy shows a change pending because of the bundle and bundle_hash values cannot be imported from the server. This causes a computed change in the revision which then causes a change in the deployment.

    # apigee_proxy.MyProxy will be updated in-place
~ resource "apigee_proxy" "MyProxy" {
  + bundle      = "proxies/hello-world/hello-world.zip"
  + bundle_hash = "7/ccJ0/1VFDm6+UyobAeYjhYQoA8kXcSm97FgX4BDIk="
    id          = "hello-world"
    name        = "hello-world"
  ~ revision    = 1 -> (known after apply)
}

# apigee_proxy_deployment.MyProxyDeployment will be updated in-place
~ resource "apigee_proxy_deployment" "MyProxyDeployment" {
    id               = "eval:hello-world"
  ~ revision         = 1 -> (known after apply)
    # (2 unchanged attributes hidden)
}
  6. To sync things up, we can either manually edit the TF state file to fix the bundle and bundle_hash values which is ugly OR we can just run apply and cause it to upload a new version of the proxy and deploy that new version. As long as your zip file matches what is currently deployed, there is no harm in deploying the same thing that is already deployed. And, of course, you can always export the current zip file from the server using the web UI so that you are guaranteed that what you redeploy is the same as what is already there. I will apply: terraform apply
  7. Check that everything is synced up with another plan: terraform plan 
    
test(main) > tf plan
apigee_proxy.MyProxy: Refreshing state... [id=hello-world]
apigee_proxy_deployment.MyProxyDeployment: Refreshing state... [id=eval:hello-world]

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your configuration and real physical resources that exist. As a result, no actions need to be performed.

davidmitchell2019 commented 3 years ago

Have you tried updating the revision of your hello world proxy?

2021/01/05 11:38:53 [ERROR] eval: *terraform.EvalApplyPost, err: Status 400: Message: Bad Request: { "error": { "code": 400, "message": "Invalid JSON payload received. Unknown name \"delay\": Cannot bind query parameter. Field 'delay' could not be found in request message.", "status": "INVALID_ARGUMENT", "details": [ { "@type": "type.googleapis.com/google.rpc.BadRequest", "fieldViolations": [ { "description": "Invalid JSON payload received. Unknown name \"delay\": Cannot bind query parameter. Field 'delay' could not be found in request message." } ] } ] } }

davidmitchell2019 commented 3 years ago

Also this proxy does contain a proxy endpoint and is a valid config but i get this error:

Error: Status 400: Message: Bad Request: { "error": { "code": 400, "message": "bundle contains errors", "status": "INVALID_ARGUMENT", "details": [ { "@type": "type.googleapis.com/edge.configstore.bundle.BadBundle", "violations": [ { "description": "The proxy must contain a ProxyEndpoint" } ] }, { "@type": "type.googleapis.com/google.rpc.RequestInfo", "requestId": "15069896738166913433" } ] } }

davidmitchell2019 commented 3 years ago

Here is an anonymised version of my proxy bundle proxy_bundle_anonymized.zip

scastria commented 3 years ago

The delay error should be fixed in the latest version 0.1.26.

scastria commented 3 years ago

Your proxy bundle is not valid. A proxy bundle has a specific directory structure. You can't just dump all xml files into 1 directory called proxy_bundle. Look at the directory structure of my hello world zip as an example. The format is described here: https://docs.apigee.com/api-platform/reference/api-proxy-configuration-reference#apiproxystructure-apiproxydirectorystructureandcontents

scastria commented 3 years ago

If you were to use the Apigee web UI and try to import your proxy bundle, you would get lots of errors as shown here: Screen Shot 2021-01-05 at 8 12 10 AM

davidmitchell2019 commented 3 years ago

Thanks that works.

Key value map operations dont seem to line up with the api docs:

https://cloud.google.com/apigee/docs/reference/apis/apigee/rest/v1/organizations.keyvaluemaps/create

Error: Status 400: Message: Bad Request: { "error": { "code": 400, "message": "Invalid JSON payload received. Unknown name \"entry\" at 'key_value_map': Cannot find field.\nInvalid JSON payload received. Unknown name \"EnvironmentName\" at 'key_value_map': Cannot find field.\nInvalid JSON payload received. Unknown name \"ProxyName\" at 'key_value_map': Cannot find field.", "status": "INVALID_ARGUMENT", "details": [ { "@type": "type.googleapis.com/google.rpc.BadRequest", "fieldViolations": [ { "field": "key_value_map", "description": "Invalid JSON payload received. Unknown name \"entry\" at 'key_value_map': Cannot find field." }, { "field": "key_value_map", "description": "Invalid JSON payload received. Unknown name \"EnvironmentName\" at 'key_value_map': Cannot find field." }, { "field": "key_value_map", "description": "Invalid JSON payload received. Unknown name \"ProxyName\" at 'key_value_map': Cannot find field." } ] } ] } }

scastria commented 3 years ago

To prevent this issue from growing extremely large, can you close this one (since I fixed your original issue) and open up a new one specifically for Apigee NG KVMs?