search

reg-viz / reg-suit

:recycle: Visual Regression Testing tool
https://reg-viz.github.io/reg-suit
MIT License
1.17k stars 101 forks source link

[reg-publish-s3-plugin] Downloading objects never finishes when there are more than 1000 objects in the download #677

Closed fukumasuya closed 7 months ago

fukumasuya commented 7 months ago

Describe the bug

It seems reg-publish-s3-plugin does not list all objects in the downloading process when there are more than 1000 objects. The plugin passes Marker parameter to ListObjectsV2Command to get all objects recursively. But ListObjectsV2Command does not support Marker and ignores it. So ListObjectsV2Command is called infinitely.. We should use ContinuationToken instead.

https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-s3/Interface/ListObjectsV2CommandInput/

https://github.com/reg-viz/reg-suit/blob/cf3ba8cbe24ecbcdab14d97617a72a7e62dfb5b6/packages/reg-publish-s3-plugin/src/s3-publisher-plugin.ts#L167-L173

Reproduced step

Steps to reproduce the behavior:

  1. Prepare an environment that has more than 1000 objects in a S3 bucket
  2. Run reg-suit with reg-publish-s3-plugin

Expected behavior

Downloading items finishes.

Actual behavior

Downloading items never finishes.