search

allenai / s2-folks

Public space for the user community of Semantic Scholar APIs to share scripts, report issues, and make suggestions.
Other
144 stars 25 forks source link

Limit and offset not working correctly on `references` endpoint #117

Closed aletar89 closed 11 months ago

aletar89 commented 1 year ago

Describe the bug When fetching using limit and offset, it's impossible to get more than 1000 results.

To Reproduce https://api.semanticscholar.org/graph/v1/paper/7b73f18fc4b72705297662ebaeb2e965ffc72781?fields=referenceCount returns 1033

https://api.semanticscholar.org/graph/v1/paper/7b73f18fc4b72705297662ebaeb2e965ffc72781/references?offset=1000&limit=33 returns {offset: 1000, data: [ ]}

https://api.semanticscholar.org/graph/v1/paper/7b73f18fc4b72705297662ebaeb2e965ffc72781/references?offset=990&limit=9 returns 9 results as expected

https://api.semanticscholar.org/graph/v1/paper/7b73f18fc4b72705297662ebaeb2e965ffc72781/references?offset=990&limit=10 returns {offset: 990, data: [ ]} and no results

Expected behavior Get all 1033 references by cycling through the offset and limit. Specifically, ?offset=1000&limit=33 should return 33 results

Additional context Seems to be working ok on the /citations endpoint. Also, this is a part of our test suite so I can say that this is a recent change (maximum a week)

hofstee commented 12 months ago

I'm having the same issue on the /citations endpoint too.

$ curl 'https://api.semanticscholar.org/graph/v1/paper/2e62d1345b340d5fda3b092c460264b9543bc4b5?fields=citationCount'
{"paperId": "2e62d1345b340d5fda3b092c460264b9543bc4b5", "citationCount": 60064}

Only one result returned, despite there being >60k citations.

$ curl 'https://api.semanticscholar.org/graph/v1/paper/2e62d1345b340d5fda3b092c460264b9543bc4b5/citations?fields=title&offset=1000&limit=1000'
{"offset": 1000, "data": [{"citingPaper": {"paperId": "eb631ea2d8ec131cb411995d7d415f597e04d12e", "title": "Determination of optimal well locations and pumping/injection rates for groundwater heat pump system"}}]}
aletar89 commented 12 months ago

I also get this in /citations though the issue is slightly different so may be an unrelated bug. In /citations it returns 1 value after 1000 while in /references it doesn't return any values at all. @wammar Is there an ETA for a fix or a suggested workaround?

rodneykinney commented 11 months ago

This has been fixed, let us know if you notice any related problems.