SciencerIO / sciencer-toolkit

A smarter way to find articles.
MIT License
9 stars 1 forks source link

Limit S2 query by terms size of requests #16

Closed ratuspro closed 2 years ago

ratuspro commented 2 years ago

While using the /paper/search entrypoint, the sum of the limit and offset must be less than 10000

The sum of offset and limit must be < 10000 (ref)

The semantic scholar provider crashes when the limit + offset are greater than 10000. Consider (1) adding a stop condition while fetching the papers and (2) constraining the maximum number of papers to 10000.