Certain consumers of the S3 API won't cope well with very large lists of files. Limit the files returned to 100 for now, and make the list be breadth-first, so you get all of the files in an outer directory which contains a small number of files and only a subset of the ones in a directory which contains a large number.
Certain consumers of the S3 API won't cope well with very large lists of files. Limit the files returned to 100 for now, and make the list be breadth-first, so you get all of the files in an outer directory which contains a small number of files and only a subset of the ones in a directory which contains a large number.