Inconsistency in DeviceFarm ListUploadsCommands

Almouro commented 9 months ago

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've checked AWS Forums and StackOverflow.
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

Using ListUploadsCommand with a type doesn't seem to properly paginate on server side

For instance:

client.send(new ListUploadsCommand({ arn: PROJECT_ARN, type: UploadType.APPIUM_NODE_TEST_PACKAGE }));

currently returns 1 result for me while in CLI:

aws devicefarm list-uploads --arn PROJECT_ARN --type APPIUM_NODE_TEST_PACKAGE

returns 17

At some point, ListUploadsCommand was actually returning 0 results for me, even though the CLI had 17

Wild guess but it feels like:

the SDK returns paginated results of all uploads
the CLI returns all results with no pagination
the filtering is done on client side, so if I don't have uploads of type APPIUM_NODE_TEST_PACKAGE in the first page then I don't get empty results

SDK version number

@aws-sdk/client-device-farm@3.387.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v18.18.0

Reproduction Steps

Create uploads with many different types

Use:

client.send(new ListUploadsCommand({ arn: PROJECT_ARN, type: UploadType.APPIUM_NODE_TEST_PACKAGE }));

Compare results with aws devicefarm list-uploads --arn PROJECT_ARN --type APPIUM_NODE_TEST_PACKAGE

Observed Behavior

Results are returned with aws devicefarm list-uploads --arn PROJECT_ARN --type APPIUM_NODE_TEST_PACKAGE

But only zero or one result is returned with client.send(new ListUploadsCommand({ arn: PROJECT_ARN, type: UploadType.APPIUM_NODE_TEST_PACKAGE }));

Expected Behavior

client.send(new ListUploadsCommand({ arn: PROJECT_ARN, type: UploadType.APPIUM_NODE_TEST_PACKAGE })); returns properly paginated results

Possible Solution

Wondering if filtering is happening server side or not

Additional Information/Context

No response

RanVaknin commented 9 months ago

Hi @Almouro ,

Thanks for reaching out. The CLI might be sending some max value to get all the results by default. Without inspecting the raw request and response of both the CLI and the SDK we cannot know for sure.

You can do that by running your CLI command with the --debug flag, and similarly you can write a small middleware that will capture the same raw logs like so:

client.middlewareStack.add(next => async (args) => {
    console.log(args.request)
    const result = await next(args);
    console.log(result);
    return next(args);
}, {step: 'finalizeRequest'})

By cross referencing the two you could see which values are actually being sent and the response.

Also, if you think the results are being paginated, you can refer to this blogpost for more info about how to paginate through responses.

Please let me know if you need any further assistance. Thanks again, Ran~

Almouro commented 9 months ago

Thanks @RanVaknin for your answer!

So the CLI SDK actually send 6 requests, each one with a different nextToken. So pretty much it paginates over all the results
This is the request the JS SDK sends:

HttpRequest {
  method: 'POST',
  hostname: 'devicefarm.us-west-2.amazonaws.com',
  port: undefined,
  query: {},
  headers: {
   // headers
  },
  body: '{"arn": <PROJECT_ARN>,"type":"APPIUM_NODE_TEST_PACKAGE"}',
  protocol: 'https:',
  path: '/',
  username: undefined,
  password: undefined,
  fragment: undefined
}

I'm getting 1 result If I send it with a nextToken, I'm getting the 2 next results

So all in all this is unexpected behavior on server side for me. If I have say 10 uploads of type APPIUM_NODE_TEST_PACKAGE, I expect my request to return the 10 results because it is not too large a payload to not be returned in a single response.

Here it seems that the server queries the first "page" of uploads in the db, then filters the results by type. Expected behavior for me would be that the server queries directly by type in the db

RanVaknin commented 9 months ago

Hi @Almouro ,

I did a small repro on my own and Im able to see more than one result returned at a time:

import { DeviceFarmClient, ListUploadsCommand } from "@aws-sdk/client-device-farm";

const client = new DeviceFarmClient({ region: "us-west-2" });

const projectArn = "arn:aws:devicefarm:us-west-2:REDACTED:project:REDACTED";
const uploadType = UploadType.APPIUM_JAVA_JUNIT_TEST_SPEC; 

try {
    const response = await client.send(new ListUploadsCommand({
        arn: projectArn,
        type: uploadType
    }))
    console.log(response)
} catch (error) {
    console.log(error)
}

Response:

{
  '$metadata': {
    httpStatusCode: 200,
    requestId: 'd03299b1-1b4d-4c2c-82cf-2b898df2a562',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  uploads: [
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:100e31e8-12ac-11e9-ab14-d663b5a4a906',
      category: 'CURATED',
      created: 2023-01-10T15:37:02.386Z,
      name: 'Default TestSpec for Android Appium Java Junit v4.0 (adds support for the latest versions of Chromedriver)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_1222_node14191_junit_android_chromedriver.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=c6a038d5bf74238d82f4ef8ad1a2b5606f2df2ee878d605b4b5005e7312ead49'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:100e31e8-12ac-11e9-ab14-d663b5a4a916',
      category: 'CURATED',
      created: 2023-11-08T19:00:00.000Z,
      metadata: '{"valid":true,"androidAmazonLinux2HostFlag":true}',
      name: 'Default Android Test Spec for Appium Java JUnit (updated 2023-11-08)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/al2_appium_java_junit_v1.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=bca7232d4103720ce1b11a8bb83388d59971b74481c809db1a21154cfa0aba63'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:100e31e8-12ac-11e9-ab14-d663b5a4a926',
      category: 'CURATED',
      created: 2023-11-17T19:00:00.000Z,
      metadata: '{"valid":true}',
      name: 'Default iOS Test Spec for Appium Java JUnit (updated 2023-11-17)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_2_ios_java_junit_v1.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=826fd284671cacf0501af482817acaaece0db6e766451c28d210cfe4c40a3560'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:100e31e8-12ac-11e9-ab14-d663bd873c81',
      category: 'CURATED',
      created: 2022-10-31T16:42:02.386Z,
      name: 'TestSpec v7.0 for iOS Appium Java JUnit (adds support for Appium 1.22.2)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_1222_node14193_java_junit_ios.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=7a89ee5f7c253031a2f417f1ee06339c320c84a94973e241299f824785d71e41'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:101e31e8-12ac-11e9-ab14-d663bd873d81',
      category: 'CURATED',
      created: 2022-04-12T12:36:17.474Z,
      name: 'Default TestSpec for Android Appium Java Junit v3.0 (adds support for Appium 1.22.2)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_1222_node14191_junit_android.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=2776a586c14b46b82a2f44773834efec8c67365965004687b42bd29814715009'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:1502522c-e4d2-4387-9c8e-95aedf626981',
      category: 'CURATED',
      created: 2020-12-02T15:37:02.386Z,
      name: 'TestSpec v6.0 for iOS Appium Java JUnit (adds support for Appium 1.19.0)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_119_junit_ios.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=51d81351ef537a8945115701513769d8e1ba2f28787779eb798da5423e06ac7b'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:245e31e8-12ac-11e9-ab14-d663bd873d08',
      category: 'CURATED',
      created: 2020-09-03T12:36:17.474Z,
      name: 'TestSpec v5.0 for iOS Appium Java JUnit (adds support for Appium 1.18.1)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_118_junit_ios.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86399&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=2573ec7c9fda4b3ec13c4b49e696fd5f891a445ace6a858bc407922650476661'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:245e31e8-12ac-11e9-ab14-d663bd873d69',
      category: 'CURATED',
      created: 2020-07-31T12:36:17.474Z,
      name: 'TestSpec for iOS Appium Java JUnit v4.0 (adds support for Appium 1.17.1)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_117_junit_ios.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=7ccfdf034123ae3cac7f3a59448fb57528bb244afc2ebe73e844ea00ae8bc3bf'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:295e31e8-12ac-11e9-ab14-d663bd873d54',
      category: 'CURATED',
      created: 2020-04-01T12:36:17.474Z,
      name: 'Default TestSpec for Android Appium Java Junit v2.0 (adds support for Appium 1.14+)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_114_115_116_junit_android.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=a6ddded7f9ccc09df766707c045039dca6a4312c57d644a9ffa83b39dc96e887'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:295e31e8-12ac-11e9-ab14-d663bd873d55',
      category: 'CURATED',
      created: 2020-04-01T12:36:17.474Z,
      name: 'Default TestSpec for iOS Appium Java Junit v3.0 (adds support for Appium 1.14+)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_114_115_116_junit_ios.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=53e524e80b7f60fb71a88a9766806e51883c043419510a00ed1c80b02d1c3444'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:4f8bd0a4-7be5-11e8-adc0-fa7ae01bbebc',
      category: 'CURATED',
      created: 2018-06-20T12:36:17.474Z,
      name: 'Default TestSpec for Android Appium Java Junit',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_java_junit_android.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=ced4091567d5a5bd69feec8028a8bd302468c23bebfb8244e438c269fa370560'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:8a6c4758-9689-11e8-9eb6-529269fb1458',
      category: 'CURATED',
      created: 2018-11-20T12:36:17.474Z,
      name: 'Default TestSpec for iOS Appium 1.9.1 Junit (Support for iOS 12)',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_191_java_junit_ios.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=7115298266c1e0160e96e216604d8bdbfe94ffe4c0a997b699776318da47ee7a'
    },
    {
      arn: 'arn:aws:devicefarm:us-west-2::upload:8a6c4758-9689-11e8-9eb6-529269fb1459',
      category: 'CURATED',
      created: 2018-06-20T12:36:17.474Z,
      name: 'Default TestSpec for iOS Appium Java Junit',
      status: 'SUCCEEDED',
      type: 'APPIUM_JAVA_JUNIT_TEST_SPEC',
      url: 'https://prod-us-west-2-uploads-testspec.s3-us-west-2.amazonaws.com/public-yaml-files/appium_java_junit_ios.yml?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231130T233257Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86399&X-Amz-Credential=AKIAUJHLTYS5AWNTRO6L%2F20231130%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=3eed5e7b70b05896cb7e6f74d313680392afe848a29a6d12e9a8b28c3dfb8d61'
    }
  ]
}

Can you share your full code snippet so we can look a bit further into it?

Thanks, Ran~

Almouro commented 9 months ago

Hi @RanVaknin, I've tried to reproduce it with a test project but can't seem to reproduce it and I don't really get how pagination is implemented:

Here's what I tried:

create an upload of type NODE
create 1000 uploads of type PYTHON
create an upload of type NODE
then ensure I can properly retrieve the 2 uploads of type NODE

This works as expected ✅

This is the snippet:

  const { project } = await client.send(
    new CreateProjectCommand({
      name: "test",
    })
  );

  if (!project) throw new Error("Project not created");

  await client.send(
    new CreateUploadCommand({
      name: `NODE Upload`,
      projectArn: project.arn,
      type: UploadType.APPIUM_NODE_TEST_PACKAGE,
    })
  );

  for (let i = 0; i < MANY_UPLOAD_COUNT; i++) {
    console.log(`Creating upload ${i}`);
    await client.send(
      new CreateUploadCommand({
        name: `OTHER UPLOAD`,
        projectArn: project.arn,
        type: UploadType.APPIUM_PYTHON_TEST_PACKAGE,
      })
    );
  }

  await client.send(
    new CreateUploadCommand({
      name: `NODE Upload`,
      projectArn: project.arn,
      type: UploadType.APPIUM_NODE_TEST_PACKAGE,
    })
  );

  const result = await client.send(
    new ListUploadsCommand({
      arn: project.arn,
      type: UploadType.APPIUM_NODE_TEST_PACKAGE,
    })
  );

  // Should have 2 results
  console.log(result);

However, what's interesting is that no pagination actually occurs. If I request all the uploads with:

const fullResult = await client.send(
    new ListUploadsCommand({
      arn: project.arn,
    })
  );

Then I retrieve ALL the results (more than 1000) which seems like way too much, and I get no nextToken

On the actual project I was facing the bug, calling:

   const fullResult = await client.send(
    new ListUploadsCommand({
      arn: project.arn,
    })
  );

returns only 484 elements, even though I have more (the request returns a nextToken)

If I iterate through all the results, I cannot find any consistency in the paging. For each page, I log:

first element created date
last element created date

page size

do {
const result = await client.send(
  new ListUploadsCommand({
    arn: PROJECT_ARN,
    nextToken,
  })
);

nextToken = result.nextToken;

console.log(result.uploads?.[0]);
console.log(result.uploads?.[result.uploads?.length - 1].created);
console.log(result.uploads?.length);
console.log("------------------------");
} while (nextToken);

Here are my results:

2023-01-10T15:37:02.386Z
2023-11-27T11:04:18.339Z
484
------------------------
2023-11-27T09:58:16.326Z
2023-11-13T15:12:44.297Z
628
------------------------
2023-11-13T11:29:27.533Z
2023-11-23T11:04:32.259Z
731
------------------------
2023-07-24T13:00:51.894Z
2023-11-14T12:02:09.702Z
739
------------------------
2023-11-14T19:05:34.528Z
2023-11-13T11:28:04.584Z
764
------------------------
2023-10-17T08:16:39.267Z
2023-10-30T01:08:57.624Z
381
------------------------

So how is the pagination implemented?

RanVaknin commented 9 months ago

Hi @Almouro ,

Thanks for the investigation. It seems like pagination was not properly implemented server side.

It seems like the device-farm service in this case only relies on nextToken, but returns inconsistent results, ie;

sometimes returns all the results without a nextToken
other times it returns some results and with nextToken

In properly supported paginatable operations, there is typically an parameter like maxResults or in this case maxUploads that will limit the results per page.

We can look at how Smithy defines what a paginator should look like:

The paginated trait is a structure that contains the following members:
Property | Type | Description -- | -- | -- inputToken | string | The name of the operation input member that contains a continuation token. When this value is provided as input, the service returns results from where the previous response left off. This input member MUST NOT be marked as required and SHOULD target a string shape. It can, but SHOULD NOT target a map shape.When contained within a service, a paginated operation MUST either configure inputToken on the operation itself or inherit it from the service that contains the operation. outputToken | string | The path to the operation output member that contains an optional continuation token. When this value is present and not empty in operation output, it indicates that there are more results to retrieve. To get the next page of results, the client passes the received output continuation token to the input continuation token of the next request. This output member MUST NOT be marked as required and SHOULD target a string shape. It can, but SHOULD NOT target a map shape.When contained within a service, a paginated operation MUST either configure outputToken on the operation itself or inherit it from the service that contains the operation. items | string | The path to an output member of the operation that contains the data that is being paginated across many responses. The named output member, if specified, MUST target a list or map. pageSize | string | The name of an operation input member that limits the maximum number of results to include in the operation output. This input member SHOULD NOT be required and SHOULD target an integer shape. It can, but SHOULD NOT target a byte, short, or long shape.WarningDo not attempt to fill response pages to meet the value provided for the pageSize member of a paginated operation. Attempting to match a target number of elements results in an unbounded API with an unpredictable latency.

In the case of device farm items is uploads, but the pageSize argument is missing basically making predictable pagination impossible.

With regards to the different ordering based on dates, this is not something that is guaranteed to happen. Sorting is usually not required in pagination so I can't fault the service in this case.

So how is the pagination implemented?

Unfortunately I'm not sure, as we on the SDK team do not have insight to service side implementations. I'll have to reach out to the device-farm team and ask them to redesign their pagination mechanism since its not functional as it is.

Your latest messages suggests something different than the initial bug report. At first you mentioned you are expecting to see 17 total uploads, but you saw 0. However in your latest message you suggested that although the results are not predictable, you are able to work with them (sometimes getting all results, and sometimes getting some with nextToken).

From this, it seems to me like you are unblocked for the specific scenario you opened the issue for?

Again, I appreciate you experimenting and clarifying everything. The more details you can provide, the better I'll be able to present this to the device-farm team to make this actionable.

Thank you very much, Ran~

Almouro commented 9 months ago

Hi @RanVaknin,

Indeed, I'm not blocked in the sense that I can bypass this issue with other less ideal means. Any chance you got an update from the server team about this?

RanVaknin commented 1 week ago

Hi @Almouro,

Sorry for the long wait. This fell off of my radar. For the service team to make this actionable can you please share any request IDs that present both type of behavior?

I tried reproducing this behavior again, and I'm not seeing any inconsistencies. When I paginate through the results, I consistently get the same number of results.

As far as the pagination goes here, since we last talked I've seen more and more services that actually use dynamic pagination. I.e; non-structured data, where as long as there is nextToken, it qualifies as a paginatable operation.

Regarding the statement about the un-ordered records that you get back, the API doesn't guarantee the records would be in any specific order.

Can you please check again if this is an issue?

Thanks, Ran~

aws / aws-sdk-js-v3