fyoorer / ShadowClone

Unleash the power of cloud
Apache License 2.0
718 stars 95 forks source link

Exception: Error 429: {"Reason":"CallerRateLimitExceeded","Type":"User","message":"Rate exceeded"}` #46

Closed monixxxx closed 4 months ago

monixxxx commented 4 months ago

exception calling callback for <Future at 0x7fd8fc0728e0 state=finished raised Exception> Traceback (most recent call last): File "/usr/lib/python3.9/concurrent/futures/_base.py", line 330, in _invoke_callbacks callback(self) File "/usr/local/lib/python3.9/dist-packages/lithops/invokers.py", line 435, in _callback future.result() File "/usr/lib/python3.9/concurrent/futures/_base.py", line 439, in result return self.__get_result() File "/usr/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result raise self._exception File "/usr/lib/python3.9/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/usr/local/lib/python3.9/dist-packages/lithops/invokers.py", line 366, in _invoke_task activation_id = self.compute_handler.invoke(payload) File "/usr/local/lib/python3.9/dist-packages/lithops/serverless/serverless.py", line 59, in invoke return self.backend.invoke(runtime_name, runtime_memory, job_payload) File "/usr/local/lib/python3.9/dist-packages/lithops/serverless/backends/aws_lambda/aws_lambda.py", line 658, in invoke raise Exception('Error {}: {}'.format(r.status_code, r.text)) Exception: Error 429: {"Reason":"CallerRateLimitExceeded","Type":"User","message":"Rate exceeded"}

random-robbie commented 4 months ago

I've had the same with a 186k line url list and 9k list.

fyoorer commented 4 months ago

This issue happens because your AWS account's concurrent lambda executions limit is low.

Request increase Concurrent executions here: https://us-east-1.console.aws.amazon.com/servicequotas/home/services/lambda/quotas