[BUG] stale-while-revalidate causes revalidation of the wrong page

[CC007]

The setting

I have a setup that uses the same URL for normal requests and for HTMX requests. This works perfectly, since I can branch on the presence of the HX-Request header on the server side.

Today I started playing with the preload extension. I artificially increased the server's response time to 1 second and saw that the preloading caused the page to be requested from the server twice.

No problem there. It just meant I had to specify a Cache-Control header in the response and settled on Cache-Control: max-age=60. This however exposed my next issue: when first loading the page through a normal request and then triggering a hx-get HTMX request, the target element of the call suddenly contained a copy of the whole webpage, instead of just the element that the HTMX request was supposed to load.

This actually made sense, because the normal request and HTMX request were sharing the same cache instance. To fix this, I added the Vary: HX-Request header to the response, so that the browser knew to cache the normal and HTMX requests separately. So far, so good.

The issue

While researching caching and the Cache-Control header, I found the stale-while-revalidate value.

The stale-while-revalidate response directive indicates that the cache could reuse a stale response while it revalidates it to a cache.

In the name of a responsive website, I of course had to try it out. It works well for normal requests. The problem however came when I tried to use this for HTMX requests. This was especially true when using the preload extension.

Expected

• Whenever a stale page is requested by HTMX within the stale-while-revalidate time window, I expect HTMX to load the response from cache. After that, I expect a revalidation to be triggered for that HTMX request (with the HX-Request header).

• Whenever a stale page is preloaded by the HTMX Preload extension within the stale-while-revalidate time window, I expect HTMX to trigger a revalidation (with or without "loading" the page from cache) for that HTMX request (with the HX-Request header).

  Actual

  When I trigger a preload on a link that has a hx-get attribute, I get 2 network events:

The second one however isn't triggered by HTMX, but is triggered by the browser to revalidate the request. The request headers for this request don't contain the HX-Request header:

The result

When the link is actually clicked, the cache is still stale and the browser will actually send the HTMX request, rather than using the stale cached value.

Also, since the revalidation triggered the caching for the normal http request, which returns the whole page, it wastes precious bandwidth and server/client time, since only the revalidation of the HTMX request was necessary.

Possible solution

It would be nice if any request headers that were provided in the Vary response header (here HX-Request) would also be used when revalidating that request.

Additional info

HTMX version: 1.9.12 Browser: Mozilla Firefox 125.0.1 Example repo: https://github.com/CC007/htmx-poc

[CC007]

It seems that MS Edge does correctly send along the headers from the previous request (and more than just the headers from the Vary response header):

Using Microsoft Edge 124.0.2478.51

Since it might be a Firefox specific issue, I reported this to Firefox too: https://bugzilla.mozilla.org/show_bug.cgi?id=1893842