This should make the s3 input compatible with more S3 compatible backends since the timestamps are taken from the same call.
It also changes the logic to something I thought makes more sense.
It shouldn't really matter if a file has changed since listing, just if it has changed during downloading/processing.
If it had just changed during listing but before the download, the exact same file would be processed twice.
This should make the s3 input compatible with more S3 compatible backends since the timestamps are taken from the same call. It also changes the logic to something I thought makes more sense. It shouldn't really matter if a file has changed since listing, just if it has changed during downloading/processing. If it had just changed during listing but before the download, the exact same file would be processed twice.