search

samvera-labs / active_encode

Declare encode job classes that can be run by a variety of encoding services
Other
6 stars 8 forks source link

MediaConvert: Memoize lookup of AWS endpoints in #mediaconvert #99

Closed jrochkind closed 2 years ago

jrochkind commented 2 years ago

The #mediaconvert method attempted to memoize (cache) it's creation of a MediaConvert client, by using @mediaconvert ||, so it would be created the first time then re-used.

However, the lookup of AWS endpoints happened on every method call anyway. This is actually the expensive part, as it's an AWS HTTP API call. Additionally, it's rate-limited by AWS, so I was getting AWS TooManyRequests exceptions on calling #mediaconvert too many times, because each time it was doing an endpoint lookup and then throwing ou the results because it already had a client memoized!

Make sure the endpoints lookup is inside the block that gets memoized/cached.

@mbklein

jrochkind commented 2 years ago

PS: The MediaConvert docs actually warn you not to do what this PR fixes!

If you request your endpoint programmatically, do so only once in your application. Don't make a request to the public endpoint each time that you make a request to MediaConvert. Otherwise, you will reach the throttle maximum on the public API endpoint.

— https://docs.aws.amazon.com/mediaconvert/latest/apireference/getting-started.html

@mbklein can you review/merge this one please?

jrochkind commented 2 years ago

Thank you!