Problematic handling of ETags

[Phrow]

While using this package in our .NET5 WebAPI project, we ran into some unexpected behavior regarding the generation/validation of ETags, which I'd like to understand - or propose a change.

Using default settings/interface implementations, when ETags are generated for an API response, it takes into account the request uri, vary-headers and the response body - which is good. This is also perfectly fine on the first request to a controller action. On the next request to that same action, however, if the same request uri and headers are present and the ETag from the previous response is sent as "If-None-Match"-header, the response is not even generated anymore because the Middleware already recognizes the uri+headers, finds the ETag in the in-memory-store, compares it with the header and immediately returns with a 304 code.

As the in-memory-store or its records never expire and the response body is never again generated after that initial request, the client always receives the same response - even if the data in the back might have changed long ago.

What's the assumption under which this behavior would be good and correct?

Our current solution: For us, Cache-Control headers would be enough and ETags are not required, but there's no way to disable them via switch. So instead, we replaced the in-memory-store for ETags with a "No-Op" store that never saves anything and thus never recognizes any RequestKeys. In this way, the ETag check always fails and responses are generated as usual. We completely rely on CacheControl location, max-age and resulting expiry times, which provide the correct results for us.

Questions/Ideas:

• Possibility to disable ETag generation?
• What about expiry in the in-memory-store for ETags? Without explicit invalidation, this could grow endlessly, couldn't it? Actually, if those records in the store would expire automatically after the max-age time, it would also solve the problem.

Looking forward to hearing your thoughts.

[KevinDockx]

What a weird request - you're the first who effectively wants to get rid of the eTags :) Never thought someone would ask for it, but sure, I can look into it if you want.

Replacement of store key records (/invalidation) is mostly automatic. Say you're interacting with values/1. First time the backend is hit and you get back an eTag in the response headers. Next request you send is again a GET request with the "If-None-Match"-header set to the eTag: the backend won't be hit. Then, you send a PUT request to values/1, which potentially results in a change; if you send a GET request now, the backend will be hit again. You can try that out with, for example, the ValuesController in the sample project. I'll clarify this in the readme. You can find the code for that from here onwards: https://github.com/KevinDockx/HttpCacheHeaders/blob/master/src/Marvin.Cache.Headers/HttpCacheHeadersMiddleware.cs#L553

If you're updating/changing records by using an out of band mechanism (eg: a backend process that changes the data in your database, or a resource gets updated that has an update of related resources as a side effect), this obviously can't be done automatically. For those case you can use the "mark for invalidation" feature: https://github.com/KevinDockx/HttpCacheHeaders#marking-for-invalidation-v5-onwards

Hope this helps :)

[KevinDockx]

Feature: disable ETag generation.

[rachna-lad]

@KevinDockx Is there any support to disable ETag generation for particular API(s)?

[KevinDockx]

@rachna-lad : yes, that is supported via the HttpCacheIgnore attribute: https://github.com/KevinDockx/HttpCacheHeaders/blob/master/src/Marvin.Cache.Headers/Attributes/HttpCacheIgnoreAttribute.cs

Apparently I forgot to add that to the readme. I added it now :)

[rachna-lad]

Thanks, @KevinDockx. I am working with .net 6 minimal API and the filter is not supported in it. Can you let me know how I can disable ETag generation for particular minimal API(s)?

[KevinDockx]

The middleware hasn't been tested with minimal APIs yet, so I'm afraid that's currently not a supported scenario.

[rachna-lad]

@KevinDockx FYI - We got the solution of how we can disable ETag generation for particular minimal API(s) by just adding a filter in Func while setting up the route. Refer below code snippet.

app.MapPost("/todoitems", handler:[HttpCacheIgnore] async (Todo todo, TodoDb db) =>
{
    db.Todos.Add(todo);
    await db.SaveChangesAsync();

    return Results.Created($"/todoitems/{todo.Id}", todo);
});