port-labs / ocean

🌊 Ocean 🌊 is an innovative solution developed by Port to seamlessly integrate various third-party systems with our developer portal product, empowering engineers to effortlessly prioritize key features and streamline the integration process.
https://ocean.getport.io
Apache License 2.0
109 stars 39 forks source link

[Integration][GCP] | Improved rate limit handling #898

Closed mk-armah closed 1 month ago

mk-armah commented 1 month ago

Description

What - Added rate limit retry and backoff strategy to when searching for resources that the caller has cloudasset.assets.searchAllResources Why - GCP throws a 419 rate limit error which prevents resources from getting synced to port. Maximum Quota that can be set on GCP for cloud asset API calls is 400, to exceed that users are required to apply for higher quota.
How - By leveraging the inbuilt support for Retry, we are able to implement an exponential backoff strategy for rate limit handling.

Type of change

Please leave one option from the following and delete the rest: