search

envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.9k stars 4.79k forks source link

CacheFilter: Make SimpleHttpCache a production-ready in-memory cache #9860

Open toddmgreer opened 4 years ago

toddmgreer commented 4 years ago

SimpleHttpCache is a minimal proof-of-concept HttpCache implementation that pays no attention to memory usage, and doesn't ever delete entries. It should be built into a production-ready in-process in-memory cache.

martin-macrometa commented 3 years ago

Is there any guidance as to what needs to be done to resolve this?

ahharu commented 2 years ago

any update here?

toddmgreer commented 2 years ago

Resolving this will at least require implementing a cap on estimated or actual memory usage, including evicting entries and rejecting excessively large insertions. Most likely, whoever decides to be the first to use SimpleHttpCache in production will find and fix whatever issues we haven't thought of (as with all software, it's buggy until proven otherwise). If you'd like to pursue this, I'll happily support you.

bryanwux commented 2 years ago

Hi @toddmgreer , I am interested in improving the SimpleHttpCache. What do you think of leveraging the existing lru_cache lib from jwt_verify_lib/simple_lru_cache to make cache eviction work as a starting point？Or if you want to keep SimpleHttpCache simple, maybe write a new production-ready http cache implentation?

toddmgreer commented 2 years ago

Hi @bryanwux, I'm excited that you want to improve SimpleHttpCache. My hope is that it can become production-ready and stay simple enough to serve as a reference. I think jwt_verify_lib/simple_lru_cache isn't thread-safe, so you'll need to deal with that.

@capoferro, you were looking at addressing this--did you put anything into motion?

capoferro commented 2 years ago

@toddmgreer Some work was done here: https://github.com/capoferro/envoy/tree/lru_cache. Cooper Bethea volunteered to put some work into that, but it has gone stale.

capoferro commented 2 years ago

Can see the diff here: https://github.com/envoyproxy/envoy/compare/main...capoferro:envoy:lru_cache

bryanwux commented 2 years ago

Seems like it's workable right now. Why it has gone stale? @capoferro

capoferro commented 2 years ago

I haven't looked at the branch in much detail since Cooper took it over. If it works for what you need, by all means pull the branch and merge up to current main. Would love to see this get into main in some form.