search

awslabs / mountpoint-s3-csi-driver

Built on Mountpoint for Amazon S3, the Mountpoint CSI driver presents an Amazon S3 bucket as a storage volume accessible by containers in your Kubernetes cluster.
Apache License 2.0
193 stars 23 forks source link

Caching support #119

Closed turbotankist closed 8 months ago

turbotankist commented 8 months ago

/feature

Is your feature request related to a problem? Please describe. mountpoint-s3 supports caching-configuration: https://github.com/awslabs/mountpoint-s3/blob/main/doc/CONFIGURATION.md#caching-configuration If s3 has a lot of objects then list command could huge increase costs.

Describe the solution you'd like in detail allow mountpoint-s3-csi-driver to use caching

I've tried enable it but is doesn't work now.

dlakhaws commented 8 months ago

As of the v1.1.0 release for the CSI driver, caching is supported. The error you are most likely getting is that the cache directory does not exist. This is because the cache directory has to exist at the instance level (the root filesystem outside of the containers). In the current implementation, the caching directory has to be created manually, we will consider an enhancement of creating a caching directory before the mount happens.

If you are still seeing issues when the cache directory exists, please include the errors you are seeing in the driver's logs.

0x4d6165 commented 8 months ago

How do you create the cache directory? Do you have to log into the nodes manually and if so, do you just pick the node your application in question is running on?

dlakhaws commented 8 months ago

Yes, you can SSH into your instance(s) and create the directory (you can do it on just the instance that's running the driver or on both so you know the directory exists regardless of the node it's running on). Then when you point the same directory in mountOptions, it will use that as the cache directory.