statamic / cms

The core Laravel CMS Composer package
https://statamic.com
Other
4.08k stars 533 forks source link

Mangled static cache resulting in broken html #7033

Closed robdekort closed 1 month ago

robdekort commented 2 years ago

Bug description

I've seen this odd issue for a while now. It only happens on one of my sites. This site uses full static caching. Let me try to explain. The client makes edits via the CP and every now and then this results in broken pages.

Broken page See the SVG characters and repeated address in the wrong spot (bottom of the page).

Screenshot_2022-11-10_at_09 15 00

Source Might not be relevant. But it turns out this is part of the instagram icon.

Screenshot_2022-11-10_at_09 17 46

Code responsible for that icon

Screenshot_2022-11-10_at_09 18 16

Broken rendered result

Screenshot_2022-11-10_at_09 19 41

Fix I can fix this by clearing the static cache. It then looks good until it breaks again:

Screenshot_2022-11-10_at_09 19 59

Look, all neat and nice:

Screenshot_2022-11-10_at_09 21 39

Broken HTML file This is the cache file in a broken state: ca1e6a6eb70d8d75.html.zip

Extra information Even with cache available this is what the cache utility look like (0 pages in cache):

Screenshot_2022-11-10_at_09 23 57

You can see there's definitely cache stuff:

Screenshot_2022-11-10_at_09 24 25

How to reproduce

It's very hard to reproduce. I couldn't find clear steps (yet). DM me for repo access.

Logs

No response

Environment

Statamic 3.3.36 Pro
Laravel 8.83.23
PHP 8.1.10
No addons installed

Installation

Starter Kit using via CLI

Antlers Parser

runtime (new)

Additional details

No response

robdekort commented 2 years ago

For now I've disabled the full static cache and enabled half measure to see if that still triggers the issue.

robdekort commented 2 years ago

I've re-enabled full static caching and added a timestring to the HTML for debugging purposes:

Screenshot 2022-11-10 at 10 24 50

Not because I'm brilliant. It was @JohnathonKoster who suspects something.

jasonvarga commented 2 years ago

In config/statamic/static_caching.php, try increasing lock_hold_length to 1.

robdekort commented 2 years ago

Done! Curious. What does that do. I can't seem to find it in the docs.

robdekort commented 2 years ago

Unfortunately this didn't fix the issue. I just ran into it again.

Screenshot 2022-11-12 at 16 44 40
robdekort commented 1 year ago

Client informed me it just happened again. I managed to download the generated cache file and you can see the string "Generated on: TIMESTAMP" with two different timestamps in there.

jeroenimpres commented 1 year ago

@robdekort Sorry to have no solution for you, but I can tell you what the lock_hold_length does. It sleeps() for $value seconds before closing the file, to prevent other files from writing to it at the same time. I'm not sure if this actually fixes anything, because of course it will also sleep for another request

robdekort commented 1 year ago

Hey thanks for the explanation @jeroenimpres. So after writing to the html file it locks it? I can confirm a lock_hold_length of 1 doesn't help. Are you running into similar issues?

jeroenimpres commented 1 year ago

I think it might help some cases, I was a bit to quick to judge it. It 'locks' the file because it was opened by PHP, and before any writing is done it checks with flock() if there is a lock, if there it, it will not write anything. So in your case it's probably mangled before the actual writing is done.

I do not run into this myself, I was just browsing open tickets with the Cache-tag, looking for a way to not (static) cache pages when the visitor is also logged into the CP (because it now caches the peak toolbar haha).

robdekort commented 1 year ago

(Side note: If you use the toolbar in production you could consider wrapping it with the {{ nocache }} tag).

robdekort commented 1 year ago

Hey @jasonvarga. Sorry to ping you but would you willing to look into another workaround? This keeps happening and I can't solve it.

robdekort commented 1 year ago

Happening on my own site now as well. Can't pinpoint when or how.

Screenshot 2023-02-25 at 14 51 59
aerni commented 1 year ago

I've been running into this on my own website as well. Every now and then, there's a broken piece of SVG displayed at the bottom of the page. Exactly how Rob is describing it. The site uses full static caching too.

robdekort commented 1 year ago

Sorry to hear that. It's not always svgs in my case. In my latest screenshot it appears to be an html comment.

John Koster is convinced its a cache race condition.

elmanvebs commented 1 year ago

I had the same issue, but only with the multi-site. To fix it, I had to delete the affected page, deploy the changes (including app cache/static clean up), and then manually restore the page. After completing these steps, the page rendered properly. And I am not sure what caused the issue.

jasonvarga commented 1 year ago

This doesn't seem to happen when using half measure, right?

robdekort commented 1 year ago

I'm afraid I don't know.

aerni commented 1 year ago

I haven't tested with half measure. I can only confirm the issue with full measure.

jasonvarga commented 1 year ago

How frequently does it happen?

robdekort commented 1 year ago

Either multiple times a week or once every few months.

jasonvarga commented 1 year ago

Do you know if it's mangling the pages around the time they are being edited by a client? (I know that may be hard to answer.)

robdekort commented 1 year ago

It does seem like that yep!

jasonvarga commented 1 year ago

New theory is a race condition where the url is invalidated and the page is hit on the frontend at the same time.

robdekort commented 1 year ago

Makes sense. And it could very well be the client checking changes.

robdekort commented 1 year ago

Is there an update on this one or is it on hold?

jasonvarga commented 1 year ago

7722 is fixing this. Would you be able to try this PR using a Composer patch?

robdekort commented 1 year ago

Thanks for your work on this Jason. I just installed the composer patch but I get the following error upon visiting the site:

Screenshot 2023-03-18 at 14 53 34
jasonvarga commented 1 year ago

You shouldn't get that error if you have static caching enabled.

robdekort commented 1 year ago

Figures! Just pushed to production where static caching is on.

jasonvarga commented 1 year ago

I've fixed it anyway if you want to update.

robdekort commented 1 year ago

Cool. Thanks. Done!

robdekort commented 1 year ago

I can't confirm if it's indefinitely fixed but I can confirm the issue didn't occur again.

jasonvarga commented 1 year ago

I'll consider 2 weeks of silence a win.

robdekort commented 1 year ago

Hahaha. I'll find you if it breaks again ;-).

robdekort commented 1 month ago

I'm afraid I've seen this happen again, on two separate sites. First time was September 16th, 2025.

robdekort commented 1 month ago
Screenshot 2024-09-23 at 20 25 58

Here's an example.

duncanmcclean commented 1 month ago

@robdekort This issue is pretty old at this point. Are you able to open a new issue instead?

robdekort commented 1 month ago

~Seriously?~

This had me a bit annoyed at first. Sorry about that. Opened the new issue though!