API Rate limit throttle log_level

exitcode0 commented 4 weeks ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

It seems that the API rate limit log message is only printed to stdout when TF_LOG=DEBUG (Link)

Is it possible for this to be a configurable option? If this was a config option, it would make it much easier to identify when API rate limits are impacting deployment times. This would make it easier to identify when either:

Seek additional API quota
Divide my Terraform state into smaller pieces

New or Affected Resource(s)

provider "okta" {}

Potential Terraform Configuration

provider "okta" {
  max_api_capacity = 95

  # Valid options "TRACE", "DEBUG", "INFO", "WARN"
  # Default value could remain as "DEBUG" or optionally be changed to "INFO"
  max_api_capacity_loglevel = "INFO"
}

References

0000

duytiennguyen-okta commented 4 weeks ago

@exitcode0 If all you need for TF_LOG=debug to print somewhere other than stdout then you could add TF_LOG_PATH to the command. TF_LOG=DEBUG TF_LOG_PATH=TestAccOrderResource.log terraform apply is what I use when trying to see the issue. Let me know if that help

exitcode0 commented 3 weeks ago

Hey @duytiennguyen-okta What I'd like to be able to achieve is to log when API rate limit back-offs are increasing plan/apply execution times It's currently possible to infer that watching the logs as there will be a long pause between log output messages Ideally i'd like to be able to log API rate limit back-offs without logging the additional information that is less relevant for this particular use-case

duytiennguyen-okta commented 3 weeks ago

Can you show me an example of what it would look like in your opinion?

exitcode0 commented 2 weeks ago

The config could look something like this

provider "okta" {
  max_api_capacity = 95

  # Valid options "TRACE", "DEBUG", "INFO", "WARN"
  # Default value could remain as "DEBUG" or optionally be changed to "INFO"
  max_api_capacity_loglevel = "INFO"
}

As for the log message itself, something like the following would be great

data.okta_groups.aws_000000000000_admin: Reading...
data.okta_groups.aws_000000000000_admin: Read complete after 0s [id=142341]
data.okta_groups.aws_111111111111_admin: Reading...
data.okta_groups.aws_111111111111_admin: Read complete after 0s [id=831242]
Pausing for 15s for API Rate limit bucket '831BCAD8-3B01-496A-B4C4-378A01630562' - '/api/v1/grops/'
okta_app_saml.aws: Reading...
okta_app_saml.aws: Read complete after 0s [id=24215]
okta_app_saml.salesforce: Reading...
okta_app_saml.salesforce: Read complete after 0s [id=507622]
Pausing for 45s for API Rate limit bucket '3A6F5113-8401-4D95-8543-D2C3849655EC' - '/api/v1/apps/'
okta_app_saml.slack: Reading...
okta_app_saml.slack: Read complete after 0s [id=442216]
Pausing for 50s for API Rate limit bucket '3A6F5113-8401-4D95-8543-D2C3849655EC' - '/api/v1/apps/'

okta / terraform-provider-okta