Purging Litespeed Cache - Failed

[rogerdawkins]

I have Litespeed configured and working. I can see X-LiteSpeed-Cache: hit in the Response Headers and I can see the lscache folder is being used. If I do a 'Clear LiteSpeed Cache' in the CP that works fine and clears out the lscache folder.

I have 'clear caches per url' switched on. However if I update an entry I get a 'Purging Litespeed Cache' Failed message every time and the cache is not cleared.

I've had a look at the source and can see it should be writing a log file but I can't see that anywhere. Two questions; firstly what could cause the cache purge to fail, any obvious things for me to check, and secondly where should the log file be generated? I'm wondering whether the log file not being there could be related, i.e. file/folder permissions.

Roger

[jsmrtn]

Hi Roger,

Could you try saving a page to get the error, and then send me your error log over so I can take a look?

The one thing I can think of from the top of my head is that the PURGE command might not be an accepted command on your hosting, could you check with your host to see if PURGE is an authorised command?

The global purge should always work because it just recursively deletes the targeted folder, whereas the per-URL taps into the deletetemplatecaches task.

Josh

[rogerdawkins]

Hi Josh

The problem is I can't get any sort of error log. I can't find your log file. It appears that it should appear in your src folder but nothing there and I tried manually creating it in case that helped but nothing. It just appears below the left column in CP as a failed task.

There is nothing in my server logs and nothing in the Craft phperrors.log either.

I also tried looking with the Yii toolbar but couldn't see anything useful.

Let me know if you know where I can find any error info and in the meantime I'll speak to the hosting company who are very helpful so should get a quick answer.

Rog

[jsmrtn]

Thanks Roger, I’ll take a look ASAP for you but might be a couple of days before I can give you a definitive answer—for now I’d suggest using the global purge until we can work out what the solution is.

Kind regards,

Josh

On Thu, 13 Sep 2018 at 17:49, rogerdawkins notifications@github.com wrote:

Hi Josh

The problem is I can't get any sort of error log. I can't find your log file. It appears that it should appear in your src folder but nothing there and I tried manually creating it in case that helped but nothing. It just appears below the left column in CP as a failed task.

There is nothing in my server logs and nothing in the Craft phperrors.log either.

I also tried looking with the Yii toolbar but couldn't see anything useful.

Let me know if you know where I can find any error info and in the meantime I'll speak to the hosting company who are very helpful so should get a quick answer.

Rog

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/thoughtfulweb/craft-litespeedcache/issues/1#issuecomment-421075031, or mute the thread https://github.com/notifications/unsubscribe-auth/ABnc7Se6CVAJ3eaXxvGcESQ6ioNHWY--ks5uaoyIgaJpZM4WnrrU .

[rogerdawkins]

Hi Josh

Thanks, that's great.

I had a reply back from the hosting company and they said... I have just tested this command and it should work correctly on our service. so PURGE should work ok.

Let me know if you need anything else from me.

Roger

[jsmrtn]

Should hopefully be fixed with 8ab9d69.

I think you might be running PHP7.2, which introduced an incompatibility using sizeof on non-countable types (apparently the object returned from getEntryById isn't countable.

I've just cut a release, so hopefully you can update your plugin and the issue will be resolved 😄

For future reference, any errors with background tasks you'll find in storage/logs/queue.log

2018-09-15 04:19:04 [::1][1][-][error][craft\queue\QueueLogBehavior::afterError]  [3] Purging Litespeed Cache (attempt: 1) - Error (time: 0.063s): sizeof(): Parameter must be an array or an object that implements Countable
2018-09-15 04:19:04 [::1][1][-][error][yii\base\ErrorException:2] yii\base\ErrorException: sizeof(): Parameter must be an array or an object that implements Countable in /Users/joshuamartin/Documents/Websites/craft-install/vendor/thoughtfulweb/lite-speed-cache/src/queue/jobs/RunLitespeedPurge.php:64

This is also where it will output the successful log from the plugin, if you wanted to check that the PURGE requests are firing successfully!

2018-09-15 04:58:37 [::1][1][-][info][LitespeedCache] Queue URL: http://craft-install/homepage
2018-09-15 04:58:38 [::1][1][-][info][LitespeedCache] Attempted to purge URL http://craft-install/homepage and got a status code of 200

Josh

[rogerdawkins]

Hi Josh

It’s a new website on a new server and I did opt for 7.2. It never even occurred to me that it could be that that was causing the issue.

I’ll give it a go and let you know if there are any issues. Once again, thanks for the plugin and also your time looking into this.

Regards Roger

From: Joshua Martin notifications@github.com Sent: 15 September 2018 13:06 To: thoughtfulweb/craft-litespeedcache craft-litespeedcache@noreply.github.com Cc: Roger Dawkins roger@formula.co.uk; Author author@noreply.github.com Subject: Re: [thoughtfulweb/craft-litespeedcache] Purging Litespeed Cache - Failed (#1)

Should hopefully be fixed with 8ab9d69.

I think you might be running PHP7.2, which introduced an incompatibilityhttp://php.net/manual/en/migration72.incompatible.php#migration72.incompatible.warn-on-non-countable-types using sizeof on non-countable types (apparently the object returned from getEntryById isn't countable.

I've just cut a release, so hopefully you can update your plugin and the issue will be resolved.

Josh

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/thoughtfulweb/craft-litespeedcache/issues/1#issuecomment-421559339, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AoaedRWswkKEIhtCe72KAZxqMPiBjLA9ks5ubO0qgaJpZM4WnrrU.

[rogerdawkins]

Hi Josh

Just a follow up on this one. I have still struggled to get per-URL caching working but it is nothing to do with the plugin. Although the first level support people at the hosting company said PURGE was supported and indeed it appeared that it was, it turns out it wasn’t! I could issue a command line PURGE and get back a 200 Purged message but the cache item was never removed.

The main reason for the email is that the hosting company said although PURGE was not supported removing individual cache items using the “X-LiteSpeed-Purge” header (https://www.litespeedtech.com/support/wiki/doku.php/litespeed_wiki:cache:developer_guide:response_headers) would definitely work.

I was thinking that for hosting companies that do no support PURGE it would be good to have support for this alternate method in your plugin. Maybe a config setting to allow a user to choose between the PURGE method or the ‘header’ method.

I attempted to use the X-LiteSpeed-Purge header option via curl but I’m not sure that is possible or the correct way it should be used. I’m happy to do some testing but not sure where to start in terms of coding it.

Regards Roger Dawkins

From: Joshua Martin notifications@github.com Sent: 15 September 2018 13:06 To: thoughtfulweb/craft-litespeedcache craft-litespeedcache@noreply.github.com Cc: Roger Dawkins roger@formula.co.uk; Author author@noreply.github.com Subject: Re: [thoughtfulweb/craft-litespeedcache] Purging Litespeed Cache - Failed (#1)

Should hopefully be fixed with 8ab9d69.

I think you might be running PHP7.2, which introduced an incompatibilityhttp://php.net/manual/en/migration72.incompatible.php#migration72.incompatible.warn-on-non-countable-types using sizeof on non-countable types (apparently the object returned from getEntryById isn't countable.

I've just cut a release, so hopefully you can update your plugin and the issue will be resolved.

Josh

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/thoughtfulweb/craft-litespeedcache/issues/1#issuecomment-421559339, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AoaedRWswkKEIhtCe72KAZxqMPiBjLA9ks5ubO0qgaJpZM4WnrrU.

[jsmrtn]

Hi Roger,

To be frank, it would be my suggestion to find a host who can set up your web server to allow the PURGE command.

I'll add a backlog item to look into adding the alternate header for PURGE, but I can't promise it will be a speedy resolution.

Kind regards,

Josh

[rogerdawkins]

Thanks Josh. It's a new hosting company that I'm in the process of moving over to. They were recommended by Leslie at Pixel & Tonic and apart from this issue they have been excellent.

I appreciate you adding it to the backlog. Let me know if you need it testing.

Roger

[davecoggins]

Hi Josh and Roger,

I've being having a go at looking into the header options as an alternative method of purging the cache and it think this also ties in with issue #2 because on some hosts (like the one I am trying out) has the permissions set on the lscache directory such that you don't have permissions to remove directories.

If you replace all the code in destoryLiteSpeedCache function to:

$response = Craft::$app->getResponse();
$response->headers->set('X-LiteSpeed-Purge', '*');

This will add the correct header to purge the whole cache.

In a similar fashion you can purge individual urls with:

$response = Craft::$app->getResponse();
$response->headers->set('X-LiteSpeed-Purge', '/the-url-you-want-to-purge');

So far I haven't found a method a passing multiple urls in a single header, however you can pass multiple tags and get Litespeed to purge by tag

$response = Craft::$app->getResponse();
$response->headers->set('X-LiteSpeed-Purge', 'tag1, tag2');

Tags can be set in the twig template using:

{% header "X-LiteSpeed-Tag: tag1, tag2" %}

I think it would be great if we could get this working in a similar fashion to https://github.com/mmikkel/CacheFlag-Craft3

Then you could have a tag for each entry plus any other tags. So for example a blog entry could be tagged with its entryID and "blog". When the entry is saved, deleted, disabled etc the entryID tag and the "blog" tag could be purged from litespeed cache.

This gets round the issues of the native cache tag getting bogged down on large sites with lots of relations.

It would be great to know your thoughts.

Cheers Dave

[jsmrtn]

Hi Dave,

Of course I'm happy for you to fire in a PR with your modification to destroyLitespeedCache—it's a much better solution, and I wasn't aware of a lot of the options we have with Litespeed in terms of header or indeed how to use response headers like you have. Will this still work in the same way as the current clearance? That is, when you hit save in the admin, it will programatically nuke the .lscache folder, without needing to interact with the front-end, or any other page on the admin. If you're happy with how this works, get a PR in and I'll check + approve as needed!

Regards to the individual PURGE requests, I agree the current method is pretty poor, but it did fit the requirement at the time (and honestly, I wasn't particularly au fait with Craft's or Litespeeds methods for handling this). The tag solution is a lot more simple, and definitely much more future-proof, but that's a serious amount of work, and I haven't got the time as I'm no longer working with Craft so my necessity for a solution is no longer there.

Kind regards,

Josh

[davecoggins]

Hi Josh,

Thanks for the quick reply. On reflection I think it probably doesn't handle all scenarios. I think it is fine if clicking the purge all button or saving an entry in the CMS becuase the header is added to the response that comes back to the browser. If my understanding is correct this header is picked up by Litespeed cache on its way to the browser and it then takes care of any purging that needs to occur.

However I'm not sure what would happen when for example a section is re-saved in Craft. When this occurs, to my knowledge, all entries are re-saved as part of a background task. I don't know enough about the mechanics of this but I suspect it wouldn't work. There is also the case of using something like Feedme plugin where entries are saved/deleted programmatically.

One thought I had was whether the plugin had a controller end point (that was protected via an api key) that you could Curl and pass in tag params and it would give back a response with the correct headers set.

So, whenever a entry is saved/deleted/etc it would curl this endpoint with correct params. This endpoint could also be used when deploying fresh code to clear the cache as part of the deploy script.

I don't think there is anything wrong with the approach you have taken btw :) I'm sure I have come across an example on the litespeed website that advises to clear out the lscache folder in exactly the same way.

I'll perhaps reach out on the litespeed forums and also see what they think.

Cheers Dave

[jsmrtn]

However I'm not sure what would happen when for example a section is re-saved in Craft. When this occurs, to my knowledge, all entries are re-saved as part of a background task. I don't know enough about the mechanics of this but I suspect it wouldn't work. There is also the case of using something like Feedme plugin where entries are saved/deleted programmatically.

As long as those actions fire the EVENT_AFTER_SAVE_ELEMENT event, the destroyLiteSpeedCache will also be fired. See here where the function fires.

One thought I had was whether the plugin had a controller end point (that was protected via an api key) that you could Curl and pass in tag params and it would give back a response with the correct headers set.

So, whenever a entry is saved/deleted/etc it would curl this endpoint with correct params. This endpoint could also be used when deploying fresh code to clear the cache as part of the deploy script.

Broadly speaking, this pretty much already happens, though I don't think you would need to protect via an API key as the PURGE request is being sent serverside by your PHP process, so it's already going to be authenticated.

Going back on myself a little, the per-URL purge method is actually robust. I was thinking of the Craft 2 version of this plugin that was pretty naff, as that required you to name the cache key as the URL you wanted to purge! This solution pulls the URL from Craft's own internal calculations when it runs the Delete Stale Template Caches task, so it should be perfect.

I think I entirely missed your point, in that you were saying changing how this works means you wouldn't need your server to support the PURGE command!

It would still be my recommendation to have a hosting provider that supports both the PURGE command, and would give your PHP process access to the .lscache folder. Anecdotally, I've used this on various shared hosting platforms and there has never been an issue. If the server is set up to support LiteSpeed, it should be set up to support the PURGE command too, in my opinion.

[davecoggins]

Hi Josh,

I've mocked up in a twig template with what I mean:

{% set purgeKey = craft.app.config.general.custom.purgeKey %}
{% set key = craft.app.request.getParam('key') %}
{% set value = craft.app.request.getParam('value') %}

{% if purgeKey == key %}
{% header "X-LiteSpeed-Purge: " ~ value %}
{% endif %}

The above represents an endpoint you could curl:

example.com/purge?key=123&value=* to purge the whole cache example.com/purge?key=123&value=/contact to purge the url '/contact' example.com/purge?key=123&value=blog,news to purge urls tagged blog or news

The first one would be useful when deploying code and you want to clear the whole cache or if saving a global in the CMS as this will most probably effect all urls. This would be the equivalent of delete the lscache directory which is currently being used. The other downside of deleting the directory directly is if you have more than one site running off the same c-panel this would flush the cache for both sites

The second one is essentially achieving the same as the current curl being used in the plugin. The main difference is the above are using response headers to clear the cache as opposed to request headers.

The third is showing how the plugin could be extended to use tags.

In reality this would be achieved as a controller endpoint as part of the plugin but hopefully illustrates what I mean.

I also added RewriteCond %{ORG_REQ_URI} !/purge to my htacces file to make sure the endpoint didn't get cache itself. That also makes me wonder that RewriteCond %{ORG_REQ_URI} !/actions should probably be added as well. I know most plugin endpoints are POST requests so would be ignore anyway but I can think of a few plugins with endpoints that are GET requests.

Maybe I'm wrong but when using the native cache tags they need to be wrapped around the whole content to make sure they are tracking everything that might change. In my experience at least this has normally ended up with huge database tables and then long running tasks when busting the cache. This is one of the original reasons I have been interested in looking for an alternative solution.

Maybe this is different now in Craft 3...

Thank you for taking the time to discuss this.

[jsmrtn]

Not a problem Dave.

Maybe I'm wrong but when using the native cache tags they need to be wrapped around the whole content to make sure they are tracking everything that might change. In my experience at least this has normally ended up with huge database tables and then long running tasks when busting the cache. This is one of the original reasons I have been interested in looking for an alternative solution.

As far as I'm aware, it's decent enough in Craft 3. Theoretically, you could cache a tiny piece of content and that would be good enough for the plugin to tap into to clear the URL for that page, think something like {% cache %}foo{% endcache %}.

As I understand it, your idea is essentially the same idea as what the plugin does now, but instead of firing PURGE requests on an event in the admin, you're wanting to create an endpoint that the plugin fires to that has response headers to clear the cache? It sounds viable, but to me also sounds like a bit of an unnecessary abstraction when it would (in my opinion) be easier to add PURGE support to the server.

I think we've reached a logical conclusion to this thread now, so I'm going to close this now, though please do feel free to throw in some closing thoughts.

[davecoggins]

Hi Josh,

Yes, your understanding of what I was proposing is correct. The only reason I went down this train of thought was because the hosting provider I have doesn't allow to delete the lscache folder in the way that your plugin does. If I was with a host that did allow it I would have used the plugin as is (in fact I have on another host where I can delete the lscache folder :))

This is where the idea of using the response header came in and since you may need to clear the cache outside of a standard CP request (e.g. deployment) that was where the idea of having an endpoint as part of the plugin came in. To me it made sense then at this point to use that endpoint for all purge requests (with the ability to clear by tag a feature in the future)

To clarify, the host I am using does support PURGE requests it is just the deleting the folder that it doesn't.

If I have time in the future I may look into this in more detail but for the time being I won't be pursuing this further.

Thank you for taking the time to discuss this.