Related to https://github.com/lichess-org/lila/issues/12066, I was trying to use the /games/search endpoint to get the list of ids and then with the /api/games/export/_ids API downloading the results.
However, /games/search only returns 12 results at a time and has a strange rate limit policy (val cost = scala.math.sqrt(page.toDouble).toInt with a 50 budget for 5 minutes) meaning getting any results towards the end is practically impossible.
It also happens directly on the Advanced Search page, just by setting the page parameter in the URL to 64, for example, and going a few pages down. With a delay of 7 seconds before page loads, which is excruciatingly slow, I still get 429s at only 228 ids.
My suggestions:
be able to specify the page size for the JSON version of /games/search, which would then be able to return with one call at least 300 ids or something
change the rate limiting to something linear or at least capped
add a parameter or create a separate API to just return the ids, and not bother about all the other information
Siderite
commented
2 months ago
Related to https://github.com/lichess-org/lila/issues/12066, I was trying to use the /games/search endpoint to get the list of ids and then with the /api/games/export/_ids API downloading the results.
However, /games/search only returns 12 results at a time and has a strange rate limit policy (
val cost = scala.math.sqrt(page.toDouble).toIntwith a 50 budget for 5 minutes) meaning getting any results towards the end is practically impossible.It also happens directly on the Advanced Search page, just by setting the page parameter in the URL to 64, for example, and going a few pages down. With a delay of 7 seconds before page loads, which is excruciatingly slow, I still get 429s at only 228 ids.
My suggestions: