search

brianpipa / overreader-issues

A place to keep track of issues and TODOs for http://overreader.com
http://overreader.com
1 stars 0 forks source link

AWS lambdas #28

Open brianpipa opened 1 year ago

brianpipa commented 1 year ago

Might be able to leverage lambdas to both search for books at Overdrive and KU. Could be done in parallel and make the searching much much quicker, in theory.

I need to make sure that won't cost too much... "The AWS Lambda free tier includes one million free requests per month and 400,000 GB-seconds of compute time per month, usable for functions powered by both x86, and Graviton2 processors, in aggregate. Additionally, the free tier includes 100GiB of HTTP response streaming per month, beyond the first 6MB per request, which are free. "

brianpipa commented 1 year ago

Got the lambda written and deployed for KU. Need to use it in the code now.

brianpipa commented 1 year ago

successfully called the lambda from test code. Progress!

brianpipa commented 1 year ago

Ive got the KU lambda completely working and integrated. I need to wait til they approve of my limit increase before i can deploy it. Now I'll create a lambda for Overdrive book lookups.

brianpipa commented 1 year ago

I've got the KU check down to about .2s per book doing them concurrently with lambdas. I think I may be able to shave a little more time off of that by sending all the lambda calls out, then doing the library calls, then coming back to check the KU lambda results. I think I can do it similarly with the library checks...

  1. send KU lambda requests
  2. send library lambda requests
  3. check KU lambda results
  4. check library lambda results
brianpipa commented 2 months ago

Not needed for libraries since the new library AP and code is blazing fast - KU searching could still benefit