I started to use the package on daily basis and my issue is the speed of listing all files.
In my case, I have more than 4,000 files on the drive and files are being uploaded and downloaded every second.
Is there an optimal way to get the list of the files in the bucket quicker?
7 seconds is quite slow
## load package
library("aws.s3")
## code goes here
st = Sys.time()
get_bucket(bucket, max = Inf)
print(Sys.time() - st)
## session info for your system
R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8.1 x64 (build 9600)
Hi,
I started to use the package on daily basis and my issue is the speed of listing all files. In my case, I have more than 4,000 files on the drive and files are being uploaded and downloaded every second. Is there an optimal way to get the list of the files in the bucket quicker? 7 seconds is quite slow