change endpoing API resource_ibm_resource_tag

ibmappm commented 1 year ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

Affected Resource(s)

ibm_resource_tag

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please share a link to the ZIP file.

terraform.zip

Debug Output

Panic Output

we execute an apply and fail by getting resource tags

[ERROR] Error getting resource tags for: crn:v1:bluemix:public:is:eu-de-1:a/44179e2e761f424a980321e07946xxx::instance:xxx7_*** with error : Too Many Requests with module.compute.ibm_resource_tag.update["hostname"], on .terraform/modules/compute/main.tf line 86, in resource "ibm_resource_tag" "update": 86: resource "ibm_resource_tag" "update"

Expected Behavior

That the call to the APi is answered without problems

Actual Behavior

Steps to Reproduce

We were testing to restore and modify a VSI, and in this test we were modifying the profile. Attached is the execution of the pipeline again.

terraform apply 17/10 17:30 first failure �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1m[ERROR] Error getting resource tags for: crn:v1:bluemix:public:is:eu-de-1:a/44179e2e761f424a980321e07946cxxx::instance:02b7_9242127e-a431-478b-b710-fb4f19586xxx with error : Too Many Requests�[0m �[31m│�[0m �[0m �[31m│�[0m �[0m�[0m with module.compute.ibm_resource_tag.update["xxx11"], �[31m│�[0m �[0m on .terraform/modules/compute/main.tf line 86, in resource "ibm_resource_tag" "update": �[31m│�[0m �[0m 86: resource "ibm_resource_tag" "update" �[4m{�[0m�[0m
17/10 17:53 retry failed in plan �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1m[ERROR] Error getting resource tags for: crn:v1:bluemix:public:is:eu-de-1:a/44179e2e761f424a980321e07946xxx::instance:02b7_9242127e-a431-478b-b710-fb4f19586xxx with error : Too Many Requests�[0m �[31m│�[0m �[0m �[31m│�[0m �[0m�[0m with module.compute.ibm_resource_tag.update["xxx11"], �[31m│�[0m �[0m on .terraform/modules/compute/main.tf line 86, in resource "ibm_resource_tag" "update": �[31m│�[0m �[0m 86: resource "ibm_resource_tag" "update" �[4m{�[0m�[0m
18/10 08:10 Finished: SUCCESS

If you need to see the full logs we can send them privately as they contain sensitive information

Important Factoids

We encountered a random bug related to the tag API call.

We opened a case with the IBM provider and they told us a solution that we can't implement because the call is included in the provider. This is case CS3070529.

answer in this case:

Can you please use below way to retrieve resources tags that might be particularly helpful to those of you receiving frequent 429 (Too Many Reqeuests ) from the tagging api.

The preferred way to retrieve the tags attached to a resource is the Search API, as shown in the following example.

curl -s -H'Content-type:application/json' -H'Accept:application/json' -H"Authorization:Bearer $IAM_TOKEN" -X POST 'https://api.global-search-tagging.test.cloud.ibm.com/v3/resources/search' -d'{"query":"crn:"crn:v1:staging:public:codeengine:us-south:a/aa00ffbccdb34a6fbe7a6499ce091aaa:0a01030b-1234-1234-aaaa-002245318e88::"", "fields":["crn","name","tags", "access_tags", "service_tags"]}'

If the resource with the given CRN exists and you can read it, then you should get the following example output, which includes all the tag types eventually attached to the resource:

{
"items": [
{
"name": "my-codeengine-instance",
"crn": "crn:v1:staging:public:codeengine:us-south:a/aa00ffbccdb34a6fbe7a6499ce091aaa:0a01030b-1234-1234-aaaa-002245318e88::",
"tags": ["one", "two"],
"access_tags": ["atag:one", "another-tag:two"],
"service_tags": ["service-tag:one", "service-tag:two"]
}
],
"limit": 10,
"search_cursor": "..."
}
You can also use the Get tags API (https://cloud.ibm.com/apidocs/tagging#list-tags) with the attached_to query parameter to obtain the same result. However, that API is strictly rate limited and it does not return all the tag types in a single call as the /v3/resources/search does. So, its usage is discouraged.

We manage the infrastructure with Terraform and the IBM provider. This link ( https://github.com/IBM-Cloud/terraform-provider-ibm/blob/master/ibm/service/globaltagging/resource_ibm_resource_tag.go) shows the configuration. And you can see that the endpoint is controlled by the provider.

References

0000

hkantare commented 1 year ago

After our investigation and communication from global Tagging API regaridng the workaround solution

We have added a brief guide to the docs about the recommended way to [retrieve resources tags](https://test.cloud.ibm.com/docs/get-coding?topic=get-coding-ghost-faq#how-to-get-resource-tags-). That might be particularly helpful to those of you receiving frequent 429 (Too Many Requests) from the Tagging API.

We are working on adding retry to make a call to Global serach API when we receive 429 (Too Many Requests) PR is in development https://github.com/IBM-Cloud/terraform-provider-ibm/pull/4147 We will update the ETA soon

hkantare commented 1 year ago

Fixed in https://github.com/IBM-Cloud/terraform-provider-ibm/releases/tag/v1.47.1

mkrudele commented 1 year ago

Pls, do not call the GET /v3/tags/ with the attached_to flag. Use the search instead, as described here https://test.cloud.ibm.com/docs/get-coding?topic=get-coding-ghost-faq#how-to-get-resource-tags- (How do I get tags attached to a resource?).
If you get zero results from the search and you know that the resource exists and the user can read it, then add some retries.

vbontempi commented 1 year ago

@hkantare I performed some tests using ibmcloud terraform provider tag 1.47.1: the tags module is still performing GET requests on global tagging api with attached_to query parameter, it retries the same request 20 times (with a pause of 5s) in the case of a failure. Then it falls back on Global search API and it completes successfully

As suggested above by @mkrudele https://github.com/IBM-Cloud/terraform-provider-ibm/issues/4125#issuecomment-1331962452 the provider should use the Global search API only in order to retrieve the list of tags attached to a resource.

IBM-Cloud / terraform-provider-ibm