search

openSUSE / open-build-service

Build and distribute Linux packages from sources in an automatic, consistent and reproducible way #obs
https://openbuildservice.org
GNU General Public License v2.0
936 stars 440 forks source link

`/search/published/binary/id` does not paginate well #9465

Open crazyscientist opened 4 years ago

crazyscientist commented 4 years ago

Issue Description

The API endpoints

show a consistent pagination behavior controlled by the parameters limit and offset. If these parameters are not given, all results are returned.

The API endpoint /search/published/binary/id does not work this way:

Working examples

The results look like this:

<collection matches="94446">
  <binary project="openSUSE:Leap:15.1:Update" repository="ports" name="update-test-32bit-pkg" version="5.1" release="lp151.12.1" arch="armv7hl"/>
  <binary project="openSUSE:Leap:15.1:Update" repository="ports" name="update-test-optional" version="5.1" release="lp151.12.1" arch="ppc64le"/>
</collection>

<collection matches="94446">
  <binary project="openSUSE:Leap:15.1:Update" repository="ports" name="update-test-32bit-pkg" version="5.1" release="lp151.12.1" arch="ppc64le"/>
  <binary project="openSUSE:Leap:15.1:Update" repository="ports" name="update-test-reboot-needed" version="5.1" release="lp151.12.1" arch="aarch64"/>

Expected Result

The API endpoint /search/published/binary/id obeys the parameters limit and offset as the other API endpoints mentioned above.

How to Reproduce

Error 413

Query any of the following API URLs.

Everyone will return:

<status code="413">
  <summary>search limit reached</summary>
  <details>413 search limit reached</details>
</status>

Magic limit of limit

Use same query as above, but increase until API does no return error 413 any more.

This will lead to a magical value of 1065 at which the API is finally returning results.

Further Information

mlschroe commented 4 years ago

It's not so magical, it's the number of matches divided by 100. It's true that for such simple searches this internal limitation should not matter. I'll try to fix this.

Do you really want pagination when doing such searches? You can specify limit=0 to get all matches.

mlschroe commented 4 years ago

Ok, hitting this limit has to do with the search order it returns. I'll talk with Adrian to find out if we can change this to something more sane.

mlschroe commented 4 years ago

I added a workaround. The sorting order needs to be discussed anyway ;)

crazyscientist commented 4 years ago

Do you really want pagination when doing such searches?

Usually I like using pagination as it reduces the time for processing single pages. Having done a small benchmark, I noticed that the time to process one request does not depend does not depend too strongly on the limit value.

So, this is really more like a cosmetic consistency issue.

adrianschroeter commented 4 years ago

On Donnerstag, 30. April 2020, 09:19:51 CEST wrote Andreas Hasenkopf:

Do you really want pagination when doing such searches?

Usually I like using pagination as it reduces the time for processing single pages. Having done a small benchmark, I noticed that the time to process one request does not depend does not depend too strongly on the limit value.

So, this is really more like a cosmetic consistency issue.

Can we help you to make your request more efficient?

Usually this interface is used to find a specific item. It is not used for tracking entire content, we have different mechanics for that.

--

Adrian Schroeter email: adrian@suse.de

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Maxfeldstraße 5
90409 Nürnberg Germany

crazyscientist commented 4 years ago

Can we help you to make your request more efficient?

@adrianschroeter I'm certainly open for suggestions