Open DominicBM opened 9 years ago
Agreed. As a public institution, it is unacceptable to limit search returns in this manner. I have never seen a peer institution require logging in to view results -- perhaps a high volume of downloads and API calls, but not browsing and search results in our open catalog.
The step ladder of 500, 2000, and 20,000 also makes no sense, especially given one record group can easily have over 2000 series -- not to mention large-scale digitization efforts. DOCUMERICA, for instance, boast over 15,000 images that people should be able to scroll through without logging in: https://catalog.archives.gov/id/542493?q=documerica
Is there a "vital" tag to add to this discussion?
@WaxCylinderRevival: The labels are all custom, and can be created by anyone. ;-)
Would you support "vital" or other verbiage? I thought "critical" might be reserved for issues related to "critical" defects so I was shying away from creating/using that label.
Or "high priority", or something, sure. In other bug trackers there is usually a way for users to vote for an issue, but we don't have that here.
Okay, I'll add "high priority."
NASA uses Socrata as its data catalog host with a built-in SODA API. For comparison, SODA APIs support a maximum of 50,000 records per page.
http://dev.socrata.com/consumers/getting-started.html "For performance, SODA APIs are paged, and return a maximum of 50,000 records per page."
I think it's pretty clear we have a desire to substantially raise or remove the offset limit in the long term. This limit makes it impossible to ever see an entire results set, whether it is in the search UI or the API (and possibly bulk export, but I am not sure if that is the same setting). Currently this limit is set at 500 for logged-out users, 2000 for regular registered users, and 20000 for power users. ST treats this as a non-starter when brought up, but I thought we should track it here anyway.