Default browser cache settings conflict with Grav cache for assets generated by assets pipelines

[nbusseneau]

Hello,

Description

I have encountered an interesting cache issue with Grav v1.6.26 using default Grav settings, with the exception of enabling assets pipelines through system.yaml:

assets:
  css_pipeline: true
  js_pipeline: true

I noticed that on the first visit of the day, CSS and JS assets would not load properly, resulting in a broken website. Inspecting network requests revealed that several GET requests for assets were failing with 404. Upon page reload the same requests succeed with 200 and next reloads yield 304, thus re-using browser cache.

Investigation

I thought this might be due to the default cache clear job scheduled every night, and confirmed the hypothesis by reproducing the issue reliably with the following protocol (prerequisite: system.yaml above):

• Access any page from browser.
  • Page loads fine (200 or 304 in network console).
• Clear cache on Grav's end (CLI or Admin plugin).
• Access same page from browser (make sure NOT to disable browser cache, easiest way is to simply access the same URL again but by pressing ENTER and not F5, and making sure cache is not disabled via network console).
  • Page loads but without CSS/JS (404 in network console).

At first I thought this was because CSS/JS pipelines were too slow, and for some reason Grav was rendering the page and returning it the client with CSS/JS links to assets before these same assets were instantiated.

To test this, I adapted the test protocol to work with multiple clients:

• Access any page from browser on client 1.
  • Page loads fine (200 or 304 in network console).
• Access any page from browser on client 2.
  • Page loads fine (200 or 304 in network console).
• Clear cache on Grav's end (CLI or Admin plugin).
• Access same page from browser on client 1.
  • Page loads but without CSS/JS (404 in network console).
• Access same page from browser on client 2.
  • Page loads but without CSS/JS (404 in network console).

As we can see, this hypothesis was incorrect as client 2 failed just like client 1 did, even though assets were properly generated following client 1 first access.

I then realized that CSS/JS filenames generated from pipelines were stable, i.e. not changing upon cache clear. A page that has not changed in content will always have the same CSS/JS assets filenames. So I thought this might be due to an unfortunate interaction between browser cache and Grav assets cache.

To test this, I modified my system.yaml to reduce expiration time significantly:

assets:
  css_pipeline: true
  js_pipeline: true

pages:
  expires: 10

And then adapted the test protocol:

• Access any page from browser.
  • Page loads fine (200 or 304 in network console).
• Clear cache on Grav's end (CLI or Admin plugin).
• Access same page from browser immediately (before 10 seconds have passed since first GET).
  • Page loads but without CSS/JS (404 in network console).
• Replay steps 1 and 2.
• Access same page from browser, but only after browser cache is expired (after 10 seconds have passed since first GET).
  • Page loads fine (200 in network console).

Thus, it seems to confirm that the default Expires header of 7 days conflict with the daily Grav cache clear when using assets pipelines.

What I find very strange is that I could not reproduce the issue when assets pipelines were disabled: instead of 404, server correctly returns 304 and thus browser cache kicks in. This might indicate a server-side configuration issue. I will investigate further when I have some time.

Mitigation

Obvious workarounds are easy to implement. Take your pick:

• Disable assets pipelines.
• Tweak browser cache expiration (page.expires) and Grav cache clear schedule so that browser cache expiration is an order of magnitude shorter than cache clear schedule, minimizing the window overlap where browser cache has not expired yet but Grav cache has been cleared.

tl;dr

Browser cache has some sort of conflict with Grav cache for assets generated by assets pipelines. This breaks pages every time a Grav cache clear occurs before browser cache has expired, which with default settings is once a day (Grav cache cleared every night, whereas browser cache expiration defaults to 7 days).

What is strange is that it does not seem to happen when assets pipelines are disabled, even though logically there should not be any difference.

[rhukster]

My hunch from your initial description of your issue was that you are probably including a bunch of external assets in your pipeline, and the process of retrieving/compressing/minifying these is timing out or just failing. It seems likely that it is a network issue if it doesn't work 'sometimes', but then later works. Would really need to get a better idea of your asset setup. perhaps even including a copy/paste of your <head> tag with pipeline off so we can see all the assets listed? Better would be to see the Twig where you load the assets but that might not be possible as it could be spread out over multiple files.

I would suggest if you have external assets to remove those from the pipeline and try again.

[nbusseneau]

Here is a copy of all CSS and JS assets in my <head> tag:

<link href="/user/plugins/markdown-notices/assets/notices.css" type="text/css" rel="stylesheet">
<link href="/user/plugins/form/assets/form-styles.css" type="text/css" rel="stylesheet">
<link href="/user/plugins/langswitcher/css/langswitcher.css" type="text/css" rel="stylesheet">
<link href="/user/plugins/mathjax/assets/css/mathjax.css" type="text/css" rel="stylesheet">
<link href="/user/plugins/simplesearch/css/simplesearch.css" type="text/css" rel="stylesheet">
<link href="/user/plugins/prism-highlight/css/prism.css" type="text/css" rel="stylesheet">
<link href="/user/plugins/prism-highlight/css/themes/prism-base16-bright.dark.css" type="text/css" rel="stylesheet">
<link href="/user/plugins/featherlight/css/featherlight.min.css" type="text/css" rel="stylesheet">
<link href="/user/plugins/login/css/login.css" type="text/css" rel="stylesheet">
<link href="/user/themes/custom-quark/css-compiled/spectre.min.css" type="text/css" rel="stylesheet">
<link href="/user/themes/custom-quark/css-compiled/spectre-exp.min.css" type="text/css" rel="stylesheet">
<link href="/user/themes/quark/css-compiled/spectre-icons.min.css" type="text/css" rel="stylesheet">
<link href="/user/themes/custom-quark/css-compiled/theme.min.css" type="text/css" rel="stylesheet">
<link href="/user/themes/custom-quark/css/custom.css" type="text/css" rel="stylesheet">
<link href="/user/themes/custom-quark/css/line-awesome.min.css" type="text/css" rel="stylesheet">

<script src="/system/assets/jquery/jquery-2.x.min.js"></script>
<script src="/user/plugins/mathjax/assets/js/mathjax.js"></script>
<script src="/user/plugins/featherlight/js/featherlight.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script>
$(document).ready(function(){
    $('a[rel="lightbox"]').featherlight({
        openSpeed: 250,
        closeSpeed: 250,
        closeOnClick: 'background',
        closeOnEsc: '1',
        root: 'body'
    });
});
</script>

I use the Quark theme, so all of these are loaded via base.html.twig through the Asset Manager.

When enabling pipelines, here is what <head> looks like:

<link href="/assets/20d564efdecc7734496fbe04722fc399.css" type="text/css" rel="stylesheet">

<script src="/assets/8a5e312f49c642b5c40ba888e2904aaa.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script>
$(document).ready(function(){
    $('a[rel="lightbox"]').featherlight({
        openSpeed: 250,
        closeSpeed: 250,
        closeOnClick: 'background',
        closeOnEsc: '1',
        root: 'body'
    });
});
</script>

You'll note that I had already disabled externals inclusion (the third-party MathJax plugin does not work when its JS is merged, this is something I have noted for a later investigation as I'm unsure if it is a bug on Grav's end or the MathJax plugin's end).

Thus, I don't think this is a network-related issue, since there is no external exclusion at all.

One thing to note:

It seems likely that it is a network issue if it doesn't work 'sometimes', but then later works.

It is not "sometimes", I can reliably make it "not work" with the protocol above.

Are there any logs I could check, or specific parts of the code where I could insert logging / debugging dumps, to help troubleshoot the generation process?

[nbusseneau]

Oh, another thing that reinforces my hunch that it is a conflict with browser cache: as stated in OP, we can clearly see in the network frames that the site breaks because the older js and css fail with 404. My hunch was that this happens because the "cached version" does not exist anymore on the server's end.

When setting enable_asset_timestamp to true, this is even easier to spot:

[rhukster]

Well, i am really am not sure how to recreate your issue. I'm using latest Grav 1.7, and on my rather large test 'sandbox' site which has over 50 plugins, many of them have their own CSS and JS. I have pipeline enabled for both JS and CSS and even after cache clear, the pipelined files are regenerated just fine. no 404s. Firefox 80.0.1 on mac btw:

[nbusseneau]

Perhaps it originates from the HTTP server? I use nginx.

Do you have a snapshot I could use to setup an identical test server, and see if I reproduce the issue with it? Otherwise I can try to make one out of a clean Grav install.

[julienloizelet]

Seems that I have the same issue. But I guess I can't help much ... Only thing I can say is:

• I can reproduce the bug only on Firefox (102.0 ); On Chrome (103.0.5060.114), it's always ok.

How I reproduce the bug each time :

• Open a page in a Firefox tab : all is ok
• Clear the cache
• Open the same page in another tab : page is broken.`(if I refresh, all is good)

On Firefox console, I can read something like:

Failed to load for <script> element whose source is "https://mydomain.com/assets/4a0e19561080fbbee6bb0b44d326c840.js

Bug disappears only if I disable JS pipeline.

[JS-Media-Creation]

I'm so happy that I finally found a thread where this issue is mentioned. I got the same issue and yes it's just a problem when using firefox, all other browsers are working fine. Are there any ideas in the meantime how to fix the pipeline function instead of just disabling?

[nbusseneau]

No I don't think anybody ever got bothered enough to look into it further. I've been running with assets pipelines disabled forever now, it doesn't bother me :D

[JS-Media-Creation]

I think it's a shame because the function itself is really good :( Maybe then I should outsource the pipelining of assets with like webpack or something else.

[JS-Media-Creation]

I believe I’ve fixed the issue. I modified the grav-admin/.htaccess file, and since then, the bug with JavaScript and CSS pipelines enabled no longer occurs. Here is my complete .htaccess file. Feel free to try it out and see if it helps for you as well.

P.S.: I’ve enabled timestamps for media and assets in the system.yaml file.