adds a couple of strategies for going slow to reduce chance of hitting RateLimitError:
batch_size
sleep_time_in_seconds
adds eval caching so that if in event of RateLimitError, we cache results and start from where it left off next time run or arun is called.
Fixes # (issue)
Type of Change
Please delete options that are not relevant.
[x] Bug fix / Smaller change
[x] This change requires a documentation update
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
[x] Tested in e2e notebooks
[x] I stared at the code and made sure it makes sense
Description
batch_size
sleep_time_in_seconds
Fixes # (issue)
Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration