search

libopenstorage / secrets

Openstorage support for Key Management Systems
Apache License 2.0
16 stars 16 forks source link

PWX-20521: Scaling GetProvider() #57

Closed nrevanna closed 3 years ago

nrevanna commented 3 years ago 
  GetProvider() function could not scale well when we tried to scale px
  to 400 nodes. GetProvider() function was limited by api rate limiting
  on EC2 side. This improvement patch takes care of two main things:
  1. Minimizes the amount of GetMetadata() calls that we need to make.
  2. Have an exponential backoff when such a limit is seen. There is an
     additional issue to handle here which relates to the type of error
     that is returned. Client timing out and EC2Metadata error are two
     other forms of rate limit being hit for EC2 apis. This has been
     handled by adding them to the list of errors that will undergo
     exponential backoff.

What this PR does / why we need it: See description above

Which issue(s) this PR fixes (optional) Closes #PWX-20521

Special notes for your reviewer: This is part of a multi-part review. PRs opened on other repos like cloudops and porx.