Are you planning a disk-backed cache?

[tomcrane]

This question is related to https://github.com/microsoft/reverse-proxy/issues/109 and replacing ARR. Will reverse proxy caching be on the roadmap? Like ARR (or Varnish)?

[Tratcher]

Nothing specific in the plans yet.

AspNetCore already has an in-memory response cache that should be proxy compatible (it keys off response cache headers). Someone could probably adapt it to use disk storage, though the concurrency model would be trickier.

Why a disk-backed cache specifically? Capacity?

[analogrelay]

It’s fair to say it’s something on our mind though. Not on the plans for 1.0 but it should be possible to do it via the extensibility points in 1.0, and we can look at it after the first version is solid.

[tomcrane]

Thanks. Yes, it's capacity - apps that can potentially generate 100s of GBs of highly cacheable content, that today you would (if you didn't fancy a CDN) stick IIS+ARR+cache, or Varnish, or NGINX Content Caching in front of.

Would be lovely to just add app.UseSuperDiskCache() or whatever. Sounds like with reverse-proxy as a toolbox it should be easier to make a disk-backed cache, but it's the kind of thing I'd much rather use something ready-made.

Really excited by this project, btw!

(What's the etiquette here - I close the issue as answered?)

[analogrelay]

Let’s leave this open to track further caching work. We’ll put it in our post-1.0 backlog though.

[fractos]

Having the option to be disk-based would really open up a lot of use cases, I believe. E.g. an integrated, disk-based cache inside a .Net Core app means fewer moving parts for a container stack. Also means you can apply logic and smarts to it locally instead of being split between the app and whatever process is doing the caching. Stretch goal for 1.0?

[karelz]

Currently, this is not critical for 1.0. As mentioned above, once the core functionality is solid, we can prioritize it with other feature requests.

[AlgorithmsAreCool]

Nothing specific in the plans yet.

AspNetCore already has an in-memory response cache that should be proxy compatible (it keys off response cache headers). Someone could probably adapt it to use disk storage, though the concurrency model would be trickier.

Why a disk-backed cache specifically? Capacity?

@Tratcher At a high level, would someone need to just wire up an implementation of IDistributedCache to make this work? (for experimentation anyway)

[Tratcher]

The response caching middleware linked above caches in-memory. You could certainly fork that to prototype caching to disk. AspNetCore already has FileBufferingWriteStream that would be useful here. https://github.com/dotnet/aspnetcore/blob/47e21d4849cc6cb88fc9ca9c95054add1261e312/src/Http/WebUtilities/src/FileBufferingWriteStream.cs#L20

[AlgorithmsAreCool]

Hmm, I'm gonna try to carve a little time to spike an implementation using FASTER KV for small to medium size responses.

Notes for myself...

• https://source.dot.net/#Microsoft.AspNetCore.ResponseCaching/ResponseCachingMiddleware.cs
  • (the aformentioned middleware)
• https://source.dot.net/#Microsoft.AspNetCore.ResponseCaching/Interfaces/IResponseCache.cs
  • (important interface)
• https://source.dot.net/#Microsoft.AspNetCore.ResponseCaching/ResponseCachingKeyProvider.cs,53
  • (stack alloc span instead of SB?)
• https://source.dot.net/#Microsoft.AspNetCore.ResponseCaching/Interfaces/IResponseCachingKeyProvider.cs,8
  • (fixed length hash instead of varlen string keys?)
• https://source.dot.net/#Microsoft.AspNetCore.ResponseCaching/ResponseCachingMiddleware.cs,225
  • uses marker interface to distinguish between a "vary" key placeholder and an actual cache entry. Vary keys are the slow path so thats why we optimize around them.
• https://source.dot.net/#Microsoft.AspNetCore.ResponseCaching/ResponseCachingKeyProvider.cs,86
  • Vary key computation is somewhat complex... don't mess with it

[Tratcher]

RE: https://github.com/dotnet/aspnetcore/issues/40229

[nixtar]

RE: dotnet/aspnetcore#40229

Nooo, it appears its been pushed to dotnet 8 😢😢. That's quite catastrophic to our tentative roadmap.

Back to IIS+ARR/nginx we go 😭.

[maknapp]

Note that https://github.com/dotnet/aspnetcore/issues/40229 is for output caching, not response caching. These are two separate things - output caching ignores cache-control headers.

I do not see any plans to add data storage options for response caching.

[adityamandaleeka]

cc @sebastienros

[Xhanti]

Some movement here: https://github.com/microsoft/FASTER/issues/902

[mgravell]

@Xhanti probably a bit premature to get too excited there, but yes; I hope we can get a FASTER-backed local-machine larger-than-RAM cache engine suitable for distributed cache, output cache, and whatever else people might want

[Xhanti]

@mgravell understood. Early days, but even if you MS does not proceed and support it, the work you've contributed here is a good jumping off point for the next person. I am also inordinately excited for this body of work :) .This is a problem I have been thinking about for some time now

[nixtar]

I've been doing a lot of research and testing recently.

Trying to shoehorn in my own disk cache solution using various caching libs including the new hybrid cache in dotnet 9. I've had some success however none that meets our requirements.

The biggest issue with all these existing caching libs is that they are all basically byte[] under the covers. We need to be able to handle files that are gigs in size and all the existing caching mechanisms end up storing this all in memory.

We could try and do things like split up the body into chunks etc (basically write a stream over the top of the http/cache requests) but at this stage we may as well just implement the caching from scratch. I'm also not sure how that could be implemented with YARP as you would end up taking a single get request from a client and turning it into many range requests as you "stream" it through the cache pipeline.

@mgravell do you know if FASTER/Garnet supports anything like streams that could fit this use case?