search

lichess-org / lila-openingexplorer

Opening explorer for lichess.org that can handle all the variants and trillions of unique positions
http://lichess.org/analysis#explorer
GNU Affero General Public License v3.0
135 stars 34 forks source link

Multi-position endpoint #204

Closed Camsbury closed 1 year ago

Camsbury commented 1 year ago

Discord post

I am doing opening analysis using these endpoints, and very easily hitting the rate limit. The responses seem to be pretty quick, so I was wondering if we could add some multi-position endpoints.

I can PR something in time as I get my rust tooling up to date, but all it really needs to be is something like Query<Vec<LichessQuery>> instead of Query<LichessQuery> and similar for the response with maybe a simple integer limit on how many of the input vector are processed.

For reference this is useful for doing something like Stockfish does for scoring, but with win/loss%, expanding the tree to a certain number of nodes, and then pruning based on best decision-making and win/loss % weighted by probabilities of seeing the response, but even being greedy with win% (and avoiding all of the above) quickly hits rate limits in any trivial opening repertoire.

niklasf commented 1 year ago

The bottleneck for the server as deployed on explorer.lichess.ovh is disk i/o, so unfortunately saving HTTP requests would not help.

Have you considered running a local instance, maybe indexing just a few months? It needs about 3x as much disk space as the .pgn.zst archives.

Camsbury commented 1 year ago

Thanks for the response! Makes plenty of sense that the bottleneck is disk i/o, and maybe what I should do is use the API for deeper lines (where many more games will be necessary to have statistically significant percentages), but use a local db for the shallower, more common lines.