TFP 1.46.0 data.ibm_atracker_* :Service URL for region not found for some of regions: ca-tor, jp-tok, jp-osa, br-sao

syyang-in-cloud commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

Affected Resource(s)

ibmatracker*

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

provider "ibm" {
  region = var.region
}

data "ibm_atracker_routes" "available" {}
data "ibm_atracker_targets" "available" {}

Debug Output

2022-10-06T14:11:35.886Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/ibm-cloud/ibm/1.46.0/linux_amd64/terraform-provider-ibm_v1.46.0 args=[.terraform/providers/registry.terraform.io/ibm-cloud/ibm/1.46.0/linux_amd64/terraform-provider-ibm_v1.46.0]
2022-10-06T14:11:35.886Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/ibm-cloud/ibm/1.46.0/linux_amd64/terraform-provider-ibm_v1.46.0 pid=3157
2022-10-06T14:11:35.886Z [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/ibm-cloud/ibm/1.46.0/linux_amd64/terraform-provider-ibm_v1.46.0
2022-10-06T14:11:35.912Z [DEBUG] provider.terraform-provider-ibm_v1.46.0: 2022/10/06 14:11:35 IBM Cloud Provider version 1.46.0  
2022-10-06T14:11:35.912Z [INFO]  provider.terraform-provider-ibm_v1.46.0: configuring server automatic mTLS: timestamp=2022-10-06T14:11:35.912Z
2022-10-06T14:11:35.977Z [DEBUG] provider: using plugin: version=5
2022-10-06T14:11:35.977Z [DEBUG] provider.terraform-provider-ibm_v1.46.0: plugin address: address=/tmp/plugin056204679 network=unix timestamp=2022-10-06T14:11:35.977Z
2022-10-06T14:11:36.253Z [WARN]  ValidateProviderConfig from "provider[\"registry.terraform.io/ibm-cloud/ibm\"]" changed the config value, but that value is unused
2022-10-06T14:11:36.254Z [INFO]  provider.terraform-provider-ibm_v1.46.0: 2022/10/06 14:11:36 Configuring IBM Cloud Session with API key: timestamp=2022-10-06T14:11:36.254Z
2022-10-06T14:11:36.254Z [INFO]  provider.terraform-provider-ibm_v1.46.0: 2022/10/06 14:11:36 [INFO] Configured Region: jp-tok: timestamp=2022-10-06T14:11:36.254Z
2022-10-06T14:11:36.458Z [DEBUG] Resource instance state not found for node "data.ibm_atracker_targets.available", instance data.ibm_atracker_targets.available
2022-10-06T14:11:36.458Z [DEBUG] ReferenceTransformer: "data.ibm_atracker_targets.available" references: []
2022-10-06T14:11:36.458Z [DEBUG] Resource instance state not found for node "data.ibm_atracker_routes.available", instance data.ibm_atracker_routes.available
2022-10-06T14:11:36.458Z [DEBUG] ReferenceTransformer: "data.ibm_atracker_routes.available" references: []
data.ibm_atracker_targets.available: Reading...
data.ibm_atracker_routes.available: Reading...
2022-10-06T14:11:36.460Z [ERROR] vertex "data.ibm_atracker_routes.available" error: service URL for region 'jp-tok' not found
2022-10-06T14:11:36.460Z [ERROR] vertex "data.ibm_atracker_routes.available" error: service URL for region 'jp-tok' not found
2022-10-06T14:11:36.460Z [ERROR] vertex "data.ibm_atracker_routes.available (expand)" error: service URL for region 'jp-tok' not found
2022-10-06T14:11:36.460Z [ERROR] vertex "data.ibm_atracker_targets.available" error: service URL for region 'jp-tok' not found
2022-10-06T14:11:36.460Z [ERROR] vertex "data.ibm_atracker_targets.available" error: service URL for region 'jp-tok' not found
2022-10-06T14:11:36.460Z [ERROR] vertex "data.ibm_atracker_targets.available (expand)" error: service URL for region 'jp-tok' not found

Error: service URL for region 'jp-tok' not found

  with data.ibm_atracker_routes.available,
  on main.tf line 45, in data "ibm_atracker_routes" "available":
  45: data "ibm_atracker_routes" "available" {}

Error: service URL for region 'jp-tok' not found

  with data.ibm_atracker_targets.available,
  on main.tf line 46, in data "ibm_atracker_targets" "available":
  46: data "ibm_atracker_targets" "available" {}

Panic Output

Expected Behavior

Actual Behavior

2022-10-06T14:21:31.316Z [ERROR] vertex "data.ibm_atracker_targets.available (expand)" error: service URL for region 'in-che' not found

Error: service URL for region 'in-che' not found

  with data.ibm_atracker_routes.available,
  on main.tf line 45, in data "ibm_atracker_routes" "available":
  45: data "ibm_atracker_routes" "available" {}

Error: service URL for region 'in-che' not found

  with data.ibm_atracker_targets.available,
  on main.tf line 46, in data "ibm_atracker_targets" "available":
  46: data "ibm_atracker_targets" "available" {}

Steps to Reproduce

terraform apply

Important Factoids

References

https://github.com/ibm-xaas/iaas-tfp/pull/13

syyang-in-cloud commented 2 years ago

I guess this issue comes from SDK typos (probably copy & paste errors): https://github.com/IBM/platform-services-go-sdk/blob/main/atrackerv2/atracker_v2.go

func GetServiceURLForRegion(region string) (string, error) {
    var endpoints = map[string]string{
        "us-south": "https://us-south.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the us-south region.
        "private.us-south": "https://private.us-south.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the us-south region.
        "us-east": "https://us-east.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the us-east region.
        "private.us-east": "https://private.us-east.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the us-east region.
        "eu-de": "https://eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the eu-de region.
        "private.eu-de": "https://private.eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the eu-de region.
        "eu-gb": "https://eu-gb.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the eu-gb region.
        "private.eu-gb": "https://private.eu-gb.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the eu-gb region.
        "au-syd": "https://au-syd.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the au-syd region.
        "private.au-syd": "https://private.au-syd.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service in the au-syd region.
        "ca-tor": "https://us-east.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for ca-tor points to the us-east region.
        "private.ca-tor": "https://private.us-east.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for ca-tor points to the us-east region.
        "br-sao": "https://us-south.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for br-sao points to the us-south region.
        "private.br-sao": "https://private.us-south.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for br-sao points to the us-south region.
        "eu-fr2": "https://eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for eu-fr2 points to the eu-de region.
        "private.eu-fr2": "https://private.eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for eu-fr2 points to the eu-de region.
        "jp-tok": "https://eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for jp-tok points to the eu-de region.
        "private.jp-tok": "https://private.eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for jp-tok points to the eu-de region.
        "jp-osa": "https://eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for jp-osa points to the eu-de region.
        "private.jp-osa": "https://private.eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for jp-osa points to the eu-de region.
        "in-che": "https://eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for in-che points to the eu-de region.
        "private.in-che": "https://private.eu-de.atracker.cloud.ibm.com", // The server for IBM Cloud Activity Tracker Service for in-che points to the eu-de region.
    }

syyang-in-cloud commented 2 years ago

or ... It seems TFP is using the old SDK version v0.28.2. TFP needs to upgrade with the latest platform-service-go-sdk of v0.28.5

JohnPaulParkin commented 2 years ago

I believe that this issue was resolved with the SDK changes mentioned above, but will only be in the TF releases starting with 1.47.0-beta0 and later due to the timing of the SDK updates and when the TF code picks up the latest dependencies.

syyang-in-cloud commented 2 years ago

I was waiting for the PR linked with the issue, but it seems someone just pushed the fix directly to master: https://github.com/IBM-Cloud/terraform-provider-ibm/commit/13e65b3c71aacfbd2dc5fe20fcb7e2624f7e0919

I hope @hkantare and admins can setup a kind of contribution rules with proper process to create PR and reviews, but it's up to them.

SeungYeop-Yang commented 2 years ago

Hi @JohnPaulParkin , I've tested with the latest TFP 1.47.0-beta0, and it's still failing as below:

ubuntu@21dad1628e04:/iaas-tfp/04-vpc-availability$ IBM_PROD_REGION="ca-tor" ./prod_test_terraform_init-upgrade_apply.sh 
+ [[ -z ca-tor ]]
+ for region in ${IBM_PROD_REGION}
+ echo ca-tor
ca-tor
+ terragrunt workspace select ca-tor -no-color
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.38.12
+ terragrunt init -upgrade -no-color
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.38.12

Initializing the backend...

Initializing provider plugins...
- Finding ibm-cloud/ibm versions matching "1.47.0-beta0"...
- Using previously-installed ibm-cloud/ibm v1.47.0-beta0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
+ terragrunt apply -no-color -auto-approve -var-file=ca-tor.tfvars
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.38.12
...
data.ibm_atracker_targets.available: Still reading... [10s elapsed]
data.ibm_atracker_routes.available: Still reading... [10s elapsed]
data.ibm_tg_gateways.available: Still reading... [10s elapsed]
data.ibm_tg_gateways.available: Read complete after 17s [id=2022-10-12 07:23:26.729424096 +0000 UTC]

Error: service URL for region 'ca-tor' not found

  with data.ibm_atracker_routes.available,
  on main.tf line 45, in data "ibm_atracker_routes" "available":
  45: data "ibm_atracker_routes" "available" {}

Error: service URL for region 'ca-tor' not found

  with data.ibm_atracker_targets.available,
  on main.tf line 46, in data "ibm_atracker_targets" "available":
  46: data "ibm_atracker_targets" "available" {}

ERRO[0020] Terraform invocation failed in /iaas-tfp/04-vpc-availability 
ERRO[0020] 1 error occurred:
        * exit status 1

ubuntu@21dad1628e04:/iaas-tfp/04-vpc-availability$

I checked out the tag v1.47.0-beta0, and it seems go.mod was not updated yet: https://github.com/IBM-Cloud/terraform-provider-ibm/blob/420da5ab6e7a58419ac5ef178ddd56de4533c8a4/go.mod#L25 ... maybe next beta or the official release of 1.47.0 will update the fix?

SeungYeop-Yang commented 2 years ago

One other thing, as mentioned in https://github.com/IBM-Cloud/terraform-provider-ibm/issues/4097 , I don't see why this atracker serviceURL were hardcoded and routed to the different regions? I couldn't find any document to justify the changes in both SDK and TFP... is there any document?

hkantare commented 2 years ago

@SeungYeop-Yang This we need to check with Atracker team who handles the support veenar@us.ibm.com is newly onboarded who owns the support

JohnPaulParkin commented 2 years ago

@SeungYeop-Yang - the reason for the URL mapping is that ATracker is not supported in all regions at the moment. The supported regions are documented in the external cloud docs. As a result, there is no ca-tor.atracker.cloud.ibm.com endpoint to operate against when performing the operations from one of the unsupported environments.

This poses a problem when someone is attempting to leverage ATracker to deliver the global events that are normally delivered to eu-de to a different region ( such as ca-tor or jp-tok ). Presumably the region in which the terraform is being executed is one of those locations just because that's the default the user normally configures and the TF provider attempts to match the atracker.cloud.ibm.com with that region ( which results in this problem ).

IBM-Cloud / terraform-provider-ibm