HENNGE / aiodynamo

Asynchronous, fast, pythonic DynamoDB Client
https://aiodynamo.readthedocs.io/
Other
73 stars 21 forks source link

Auto paginating endpoints get throttled if passed invalid value #102

Closed jarikujansuu closed 2 years ago

jarikujansuu commented 2 years ago

For example if pass limit "foobar" to limit in count it gets throttled. At least in tests against DynamoDb Local

____________________________________________________________________________________________________________ test_count_with_limit[httpx-True] _____________________________________________________________________________________________________________

client = Client(http=HTTPX(client=<httpx.AsyncClient object at 0x105f3f4f0>), credentials=<aiodynamo.credentials.ChainCredentia...class 'float'>, throttle_config=ExponentialBackoffThrottling(time_limit_secs=60, base_delay_secs=2, max_delay_secs=20))
prefilled_table = '429ba920-a8ec-44b1-bd2b-44c17b4ee4b4', consistent_read = True

    @pytest.mark.parametrize("consistent_read", [None, True, False])
    async def test_count_with_limit(
        client: Client, prefilled_table: TableName, consistent_read: Optional[bool]
    ) -> None:
>       assert (
            await client.count(
                prefilled_table,
                HashKey("h", "h"),
                limit="foobar", #90,
                consistent_read=consistent_read,
            )
            == 90
        )

tests/integration/test_client.py:150: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/aiodynamo/client.py:824: in count
    async for result in self._depaginate("Query", payload, limit):
src/aiodynamo/client.py:999: in _depaginate
    result = await task
src/aiodynamo/client.py:945: in send_request
    async for _ in self.throttle_config.attempts():
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = ExponentialBackoffThrottling(time_limit_secs=60, base_delay_secs=2, max_delay_secs=20)

    async def attempts(self) -> AsyncIterable[None]:
        deadline = time.monotonic() + self.time_limit_secs
        for delay in self.delays():
            yield
            if time.monotonic() > deadline:
>               raise Throttled()
E               aiodynamo.errors.Throttled

src/aiodynamo/models.py:238: Throttled
dimaqq commented 2 years ago

it would be nice to determine what the "inner" exception was... or what the server response was.

ojii commented 2 years ago

it would be nice to determine what the "inner" exception was... or what the server response was.

that's the bug I think. actual errors (such as sending bad params) are incorrectly considered "retryable" errors. it should fail on the first iteration.

dimaqq commented 2 years ago

I think I've just ran into this myself... the actual error being "credentials no longer valid" in my case :orz: