search

hashgraph / guardian

The Guardian is an innovative open-source platform that streamlines the creation, management, and verification of digital environmental assets. It leverages a customizable Policy Workflow Engine and Web3 technology to ensure transparent and fraud-proof operations, making it a key tool for transforming sustainability practices and carbon markets.
Apache License 2.0
99 stars 129 forks source link

Pagination API endpoint returns lack crucial details #4085

Closed mwandres-hashgraph closed 2 weeks ago

mwandres-hashgraph commented 1 month ago

Problem description

When calling an api endpoint that uses pagination, the pagination is working but does not provide enough information to automatize the work with the response.

Step to reproduce

Call any api endpoint that supports pagination.

Expected behavior

To allow automated processes to use the response of a paginated api endpoint, please enhance the response and add the following fields to it: a) "page" - Holding the current page number. b) "pageSize" - Holding the number of items of the current page number. c) "hasNextPage" - Defines whether there is a next page or not d) "hasPreviousPage" - Defines whether there is a previous page or not. e) "filteredPagesCount" - The total number of pages that have been returned according to the set filters. f) "filteredCount" - The total number of items that have been returned according to the set filters. g) "totalCount" - The total number of items in existence.

anvabr commented 3 weeks ago

@mwandres-hashgraph We started working on this, but are confused with the exact requirements as our understanding contradicts what we thought common practices for paging (we think). Can you please clarify what exactly do you mean by the following points:

b) "pageSize" - Holding the number of items of the current page number.

Is this the number of items on this current page for which point a) would return the number? We are not sure because in Guardian implementation all pages are of the same size, so it's not 'current' page but just general 'per page'. 

c) "hasNextPage" - Defines whether there is a next page or not
d) "hasPreviousPage" - Defines whether there is a previous page or not.

Why should the server be sending this every time? The client has the information to calculate these by itself. 

e) "filteredPagesCount" - The total number of pages that have been returned according to the set filters.

Once again, the client has this information as it is receiving pages.

f) "filteredCount" - The total number of items that have been returned according to the set filters.

Having the numbers of pages, and the number of items per page this should be easily calculable-able by clients.

g) "totalCount" - The total number of items in existence.

Do you mean the total number of items which have been 'selected' by the filters and would be returned if the client stepped through all the pages?

mwandres-hashgraph commented 3 weeks ago

@anvabr , please find my response below:

Is this the number of items on this current page for which point a) would return the number? We are not sure because in Guardian implementation all pages are of the same size, so it's not 'current' page but just general 'per page'.

Example: we have 17 items and the page size is 10. We will receive 2 pages. 1 page with 10 and one page with 7. The name might be a little bit misleading, but I guess this example makes clear what we are talking about. I agree that the size of the page will always be able to hold 10 items, but we are looking for the number of actually returned items here - which might differ from 10.

Why should the server be sending this every time? The client has the information to calculate these by itself.

If you iterate over each page, this makes it more handy. And judging by the already existing overhead (for example when querying VCs) I think this could actually add some value when handling the integration of the endpoint.

Once again, the client has this information as it is receiving pages.

The client should receive this information directly from the call and should not be required to add any additional code that queries array length or the liking.

Having the numbers of pages, and the number of items per page this should be easily calculable-able by clients.

Yes, but you have this information already. Why not return it? And without the information of how many items the last page contains, this is currently not calculatable.

Do you mean the total number of items which have been 'selected' by the filters and would be returned if the client stepped through all the pages?

This would be al items in existence if no filters would have been applied.

anvabr commented 3 weeks ago

The issue with returning all this information is that, contrary to your statement, the server does not have all of it already - some of these require additional DB queries, which puts stress on the server, and correspondingly also slows things down for the client.

anvabr commented 2 weeks ago

We've added the requested data, except for the below as these require additional (slow running) DB queries:

e) "filteredPagesCount" - The total number of pages that have been returned according to the set filters. f) "filteredCount" - The total number of items that have been returned according to the set filters.

Please see commit into develop branch https://github.com/hashgraph/guardian/commit/7c1ac9240c177274508da0679431da47b8911142 for reference.