search

versity / versitygw

versity s3 gateway
https://www.versity.com/products/versitygw/
Apache License 2.0
180 stars 23 forks source link

[Feature] - Cache backend files/blobs/objects #881

Open polyrabbit opened 2 weeks ago

polyrabbit commented 2 weeks ago

Describe the solution you'd like

Would you consider caching the backend objects in the gateway? This will benefit repeated read scenarios a lot. Another use case is to warm-up the cache in advance if user knows what to read next, so that user can get a better read performance.

benmcclelland commented 2 weeks ago

The main issue with read cache is when load balancing multiple gateways. If a cached object gets overwritten through a different gateway, there is no way to actively invalidate the cache on the other gateway. Maybe this could be an option for single gateway use cases though.

polyrabbit commented 2 weeks ago

Right, overwriting scenarios will cause the inconsistency issue. How about making the cache feature optional? If the user knows the data is immutable, they can enable this feature. I suppose most users using object storage donot overwrite their objects frequently - this will be a appealing feature for them.