search

galaxyproject / galaxy

Data intensive science for everyone.
https://galaxyproject.org
Other
1.42k stars 1.01k forks source link

UX should reflect zenodo pagination only supports first 10k entries #19121

Open martenson opened 3 weeks ago

martenson commented 3 weeks ago

See https://zenodo.org/api/records?size=25&page=401 https://zenodo.org/api/records?size=100&page=101

Currently we allow going to pages that we have no chance of resolving. This yields an error that you can recover from only by closing the dialogue.

Screenshot 2024-11-08 at 1 35 25 PM

I did not find this described in the docs but seems to be true across invenioRDM instances.

  File "/Users/marten/devel/git/galaxy/lib/galaxy/files/sources/invenio.py", line 473, in _ensure_response_has_expected_status_code
    raise Exception(
Exception: Request to https://zenodo.org/api/records?size=25&page=164378 failed with status code 400: Invalid querystring parameters.
galaxy.webapps.base.api INFO 2024-11-08 13:40:22,156 [pN:main,p:89383,tN:MainThread] MessageException: Problem listing file source path zenodo://zenodo/
davelopez commented 3 weeks ago

Does this look like a bug in Zenodo or InvenioRDM API? :thinking:

martenson commented 3 weeks ago

@davelopez to me it feels like a deliberate undocumented choice...but yeah, hard to say

davelopez commented 2 weeks ago

I don't know. I think I will create an issue upstream to ask for clarification. If the API returns a total number of hits > 10k, you should be able to reach those records.

davelopez commented 2 weeks ago

Found this: https://github.com/zenodo/zenodo-rdm/issues/217