Closed AstrakhantsevaAA closed 1 month ago
Name | Link |
---|---|
Latest commit | 559f344d61ee53f63679d0cef9ae45fc892a5273 |
Latest deploy log | https://app.netlify.com/sites/dlt-hub-docs/deploys/66fbeecc09c7a20008b93b35 |
I set paginators explicitly, because they are not detected automatically for some reason
Can you reproduce it? If yes, let's create an issue for this and fix it. (but we still can explicitly define the paginators of course)
If we want to explicitly define pagination, we'd need to update the tutoral as it says that we didn't explicitly specify pagination: https://dlthub.com/docs/tutorial/rest-api#pagination
@burnash
Can you reproduce it? If yes, let's create an issue for this and fix it.
Yes, I can, unfortunately, for both sources :( Let's create an issue then.
Yes, I can, unfortunately, for both sources :( Let's create an issue then.
@AstrakhantsevaAA I think it's not an actual error, but a not very good warning message. Most likely we need to change the message. This Fallback pagiantor
warning comes from the child (dependent) resource and informs that there's no paginator on this page.
@burnash yes, I see it is just a warning, but also it is the cause of the rate limit error. Not sure why it is happening, but if i add pagination explicitly then the data loads without any problem and rate limit errors. It seems like it keeps requesting one page until the rate limit error, so you can see the list of these warnings (with pagination it takes 1-2 requests).
created the issue: https://github.com/dlt-hub/dlt/issues/1915
users have some issues with dlt init rest_api duckdb command, I set paginators explicitly, because they are not detected automatically for some reason. also reduced the amount of data for github api (request data for a week, not for month) because of rate limits