Pageable footprint

[yannbertrand]

As it was asked in Sails, it would be great to have a footprint to manage pages of data.

[tjwebb]

Yea this is a good idea. A mistake from Sails.js -- thanks for not letting me repeat it!

I think we have two options here, and we can pick one or more to implement:

1. Implement a separate count endpoint, of the form /{model}/count? ...
2. Return an X-Total-Count header on regular footprint queries
3. Advantage: no need to make a separate request
4. Disadvantage: more load on the database

They'd both be configurable, and not required. Thoughts?

[jaumard]

I think both can be useful depends of the situations. /{model}/count? is useful when we only need a number and X-Total-Count is useful when we need the total and the data. So my opinion is to have both :) and we can disable X-Total-Count with a /model/find?nocount=true for example or enable it /model/find?count=true depends if by default we want to enable/disable it.

[animedbz16]

I would recommend using Content-Range or X-Content-Range or make this name configurable.

See my issue which is also linked in the sails issue described above.

https://github.com/balderdashy/sails/issues/1730

On 4:07am, Thu, Dec 24, 2015 Jaumard notifications@github.com wrote:

I think both can be useful depends of the situations. /{model}/count? is useful when we only need a number and X-Total-Count is useful when we need the total and the data. So my opinion is to have both :) and we can disable X-Total-Count with a /model/find?nocount=true for example or enable it /model/find?count=true depends if by default we want to enable/disable it.

— Reply to this email directly or view it on GitHub https://github.com/trailsjs/trails/issues/34#issuecomment-167073227.

[tjwebb]

Content-Range is specifically for the number of bytes. Is there some other precedent to use it for indicating number of database records?

[animedbz16]

The RFC documentation indicates that bytes is one of the unit types, but that there can be application specific types defined:

Content-Range: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.16 Range Units: http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.12

The main reason, I brought this up in Sails was at the time we were working with the Dojo Framework and their JSON API needed to be aware of these Content Ranges so their Data Grid could correctly request paginated data via HTTP headers Range / Content-Range. https://dojotoolkit.org/reference-guide/1.10/dojo/store/JsonRest.html#id7

We have been migrating away from Dojo and are pretty much exclusively using Angular now, so we aren't faced with those issues as much, but I do still think that it would be nice to always have this content range / pagination data returned for requests so you can always know the total count of assets that matched the search query and know what range you are currently looking at.

If you also want to take a look at Heroku, the use a similar way makes use of the Content-Range https://devcenter.heroku.com/articles/platform-api-reference

[yannbertrand]

Thanks for these good advices @animedbz16.

I think https://github.com/trailsjs/trails/issues/35 is related.

[jacqueslareau]

In my experience, you almost always need the total count for client side pagination. Having a second endpoint for count is only moving the complexity to client code (need to generate 2 requests all the time). Maybe that will go away with HTTP2, but for now I think it should be highly configurable.

I think clients should be able the request metadata either by a query parameter (like @jaumard said) or http header. It could also be a different endpoint if you can find something that makes sense like /api/usersAsArray to disable count.

[tjwebb]

Scheduled for v1.0 beta

[scott-wyatt]

Closing as released, please feel free to reopen!