digital-science / dimcli

Python client and CLI for scientometrics and research analytics using the Dimensions API.
https://digital-science.github.io/dimcli/getting-started.html
MIT License
43 stars 6 forks source link

Skip changes limit if I want to start extraction from hits over 50k #79

Closed marton-balazs-kovacs closed 11 months ago

marton-balazs-kovacs commented 1 year ago

When I use the iterative query to look up results starting over 50k which is the API limit, the iterative query function rewrites the hard-coded limit value. Thus, it is not possible to start the iteration over the API limit. I suspect the following code chunk is responsible for the behavior:

MAXLIMIT = maxlimit or 50000 flag_last_round = False if skip + limit >= MAXLIMIT: flag_last_round = True if skip + limit > MAXLIMIT: limit = MAXLIMIT - skip

lambdamusic commented 11 months ago

The Analytics API has a built-in upper limit of 50k records for all queries.

So in dimcli auto-iteration utility this is enforced, to avoid triggering an API error.

You can test it out via a simple query, by setting the skip parameter at 50k:

image

PS Let me know if this does not answer your question..