Possible default limit in output rows

[owenlittlejohns]

Nice API! I am using it to update some legacy code, which will now be about two orders of magnitude faster, so thanks!

I am using the /bigquery endpoint, and sending a list of approximately 4000 bibliographic codes. I set rows in the input parameters to the number of bibcodes (rows: 4129). In the response header the number of rows is now limited to 2000, which is the number of elements in the response_json['response']['docs'] list. The contents of these results are otherwise as expected. Here is the response JSON:

{
    "responseHeader": {
        "status": 0, 
        "QTime": 83,
        "params": {
            "q": "*:*",
            "fq": "{!bitset}",
            "rows": "2000",
            "fl": "bibcode,citation_count,first_author,author",
            "wt": "json"
        }
    },
    "response": {"start": 0, "numFound": 4129, "docs": [...]}
}

I noticed that numFound corresponded to the expected number of output rows, so I wondered if there was a hardcoded limit on the number of returned rows, somewhere after Solr performs the query, set to 2000? It seems a bit strange to have a limit only on the amount of data returned in the response, not also applying to the input to Solr. Should I be limiting my queries to 2000 bibcodes, or should I be expecting all 4129 results in the HTTP response?

Thanks in advance.

[romanchyla]

Yes, you guessed correctly - we limit the number of results returned per one request. You can use start=2001 and start=4001 to retrieve the rest. Or, best, as you proposed - limit each query to 2000 bibcodes. The last solution is usually slightly more efficient - so we like that more :)

[owenlittlejohns]

Thanks, good to know! I'll limit my queries to <2000 bibcodes in future. That should save some resources at your end.