Issue: The basic AWS S3 connection interface doesn't support listing files in an AWS bucket when the number of files exceeds a threshold (possibly 1000).
Solution: Need to implement continuation tokens for AWS request responses to handle this scenario.
In practice, current version of VDMS never does a full listing of the bucket, but this is still an issue we should look into as we add more resiliency and recovery mechanisms over time which may wish to do so.
Issue: The basic AWS S3 connection interface doesn't support listing files in an AWS bucket when the number of files exceeds a threshold (possibly 1000).
Solution: Need to implement continuation tokens for AWS request responses to handle this scenario.