solracsf
commented
3 months ago If you use external storage in Nextcloud, you'll notice it's not the fastest. There's a lag when you navigate directories, file operations take a few moments.
I don't expect too much change using Directory Buckets. Most of the lag comes from the (HTTP) verbose between Nextcloud and the service, which will not be avoided.
The problem
If you use external storage in Nextcloud, you'll notice it's not the fastest. There's a lag when you navigate directories, file operations take a few moments.
Describe your solution
Less than a year ago, AWS introduced "S3 Express One Zone" storage or "Directory Buckets". In exchange for reduced availability and reduced redundancy, you get faster file operations and (possibly) lower costs.
Instead of being replicated across multiple zones, directory buckets are in a single zone. According to AWS, directory buckets are up to 10x faster than standard S3 buckets and may reduce costs by up to 50%.
However, note that "Directory Buckets" do not support versioning.
Code changes
I don't anticipate big changes for the current AWS S3 bucket module. The library used with AWS may need to be updated. "Directory Buckets" prefer "session-based" authentication to get the full lower-latency benefits. So, extra AWS calls may need to be added to create sessions.
I did try connecting a "Directory Bucket" to Nextcloud 29 thinking there was a chance it might "just work", but I had no luck. These buckets use a different naming scheme, and Nextcloud 29 is complaining about the "invalid" naming scheme.
Additional context
AWS "Directory Buckets" docs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html
Copying objects looks pretty much the same: https://docs.amazonaws.cn/en_us/AmazonS3/latest/userguide/directory-buckets-objects-copy.html#directory-bucket-copy-sdks
More info in this blog: https://www.pulumi.com/blog/amazon-s3-express-one-zone/
Directory Buckets (Express) vs Standard S3 Buckets