Pagination is not working

[fblenkle]

Hi,

in a project using FSXA API I was confused why my fetching logic did not work. Both fields 'totalPages' and 'size' were undefined after inspecting result of fetchByFilter. After watching source code and comparing with CaaS Result JSON, it's obvious: The result JSON does not contain ANY information how many items result for query. There are no properties '_total_pages' or '_size'!

https://github.com/e-Spirit/javascript-content-api-library/blob/2e8aa5744ed2b4b4992bc8a29a26bcd724311fd0/src/modules/FSXARemoteApi.ts#L656

Here is an (anonymized) result of CaaS:

{
  "_id": "2a3819fe-f4ca-4f61-a5eb-34a9c1910433.preview.content",
  "_etag": {
    "$oid": "63033cd2f7bd9c3c604baeac"
  },
  "streams": [
    {
      "uri": "crud",
      "stages": [
        {
          "_$match": {
            "_$or": [
              {
                "operationType": "insert"
              },
              {
                "operationType": "update"
              },
              {
                "operationType": "replace"
              },
              {
                "operationType": "delete"
              }
            ]
          }
        },
        {
          "_$project": {
            "fullDocument::_id": 1,
            "fullDocument::fsType": 1,
            "operationResult": 1,
            "documentKey": 1,
            "operationType": 1
          }
        }
      ]
    }
  ],
  "_embedded": {
     "rh:doc": [

     ]
  }
  "_returned": 5,
  "_links": {
    "crud": {
      "href": "/<project-name>/2a3819fe-f4ca-4f61-a5eb-34a9c1910433.preview.content/_streams/crud"
    }
  }
}

[deangite]

Hello, if you can look at the line https://github.com/e-Spirit/javascript-content-api-library/blob/2e8aa5744ed2b4b4992bc8a29a26bcd724311fd0/src/modules/FSXARemoteApi.ts#L602 there is an early return if there are no items returned from the CaaS.

[fblenkle]

The case is I get results, but CaaS result does not contain _total_pages, so line 656 returns undefined

[deangite]

Can you please provide what do you get from CaaS when you get results?

[fblenkle]

The result CaaS content is in my first comment

[neo-reply-lukas]

Could you provide the code line of the fetch? I guess you do not provide the additional Argument "count". Data about pagination is only added if count is set to truthy, as it has performance implications for the underlying database. See: https://github.com/e-Spirit/javascript-content-api-library/blob/9d370c538b3278f005603afd38463a6fa8c170f5/CHANGELOG.md?plain=1#L524

[fblenkle]

Thanks a lot, version 10.16.2 with additionalParams solved the problem.

[nico-mcalley]

@fblenkle sorry for closing the issue that early. We missed the open question from @neo-reply-lukas. In general we don't recommend to use the "count" parameter, since it slows down request times.

We encourage developers to use more "dynamic" pagination approaches when you don't know exactly how many pages exist - or e.g. infinite scrolling.

Your question is a good motivation to write documentation for that topic, so thank you for bringing it up 😄.

[fblenkle]

OK. Internaly we fixed this issue last week with a "do-while" loop (first time in my life ^^) as long as resulting items is lower pagesize and started discussion to change it. But after your comment I assume we will leave it.

[neo-reply-lukas]

A performant approach would be to only ask about pagination data only on the first fetch and use that info for all subsequent fetches.

[jodeleit-es]

..or to request it lazy when the request delivers exactly the same amount of hits as the page-size