Future of W3 Total Cache and WP Super Cache in EE [feedback-request]

[rahul286]

We use only nginx fastcgi_cache for full page caching i.e. wpfc option in ee site create command.

WP Super Cache is only used for full page caching so we never used it.

W3 Total Cache is generally used for many things, but have replacement already for few features and planned replacement for few other features.

1. Full page cache (Nginx fastcgi_cache is replacement)
2. Object Cache - redis and memcache - both have object-cache.php (testing is underway)
3. Database Cache - not required. I feel this is "over"caching. MySQL query cache and Object Cache together does decent job.
4. CDN - Origin/Pull CDN can be done using nginx. For S3-style CDN, many plugins exist.
5. Minify/Combine - Can be done in pagespeed but pagespeed has open issue. But there are wordpress plugins for this and one of them wp-minify - https://wordpress.org/plugins/wp-minify/ worked nice for me in past.

So we want to know, if we drop support for above 2 plugins in near feature, will it affect you? If yes, how?

One way is to move these options to ee plugins and let those who need them, manage it.

Please provide your feedback. Having less options will help us focus on improvements.

[anandslab]

I would not mind trying an environment without W3TC (as I said in the other thread). At the same time I also hate to add 3-4 plugins to replace one (object, minify, nginx helper, and possibly cdn). Difficult decision. I like the caching scheme you listed. Off-loading caching from WP would be great. Right now I have W3TC for object (memcache), minify, and CDN and FastCGI cache for page. I do plan to move to Redis. Just with page cache offloaded from W3TC to Nginx fastcgi cache i already see a bit of improvement. But once again, if I have to add multiple plugins to replace one that might be a deal breaker for me.

[cm896]

I'd rather handle these tasks outside of WP without the plugin to control and think it's possible using the object-cache.php. I have disabled W3tc and hadn't noticed any issues.
I also prefer Redis to the memcache just for the toolset/browser. I hadn't noticed any performance difference between the two. I'd rather pagespeed handle the cdn and minifying, but during our testing decided against the cdn as it offered no benefit, but rather a detriment to our overall score. Allowing pagespeed to handle the image optimization has improved our gtmetric score from 86% to 94%. Also recommend enabling SPDY if you have an SSL site. We are running full ssl sites due to the latest google ranking algorithm bump, and SPDY is a great solution to the potential handshaking hit of SSL.

So in the end, I removed one heavy multi functional plugin, and use the light weight object-cache.php file and pagespeed to do the work.

[puikinsh]

I am using W3 Total Cache for CDN, Database and Object cache.

If you would cover in tutorials how we could say goodby to W3 Total Cache and keep CDN in place and add object cache on top of it that I wouldn't mind dropping this plugin at all. If we can maintain the same or get even better performance then why not to drop support for plugins and leave everything for server to handle.

[xkidro]

I haven't been able to achieve same minify results with pagespeed (w3 total cache makes it all 1 line) pagespeed doesn't do it like w3 total cache(maybe I just misconfigured, I would be glad if anyone can give me their pagespeed config) also I haven't been able to combine files with pagespeed as well(probably my mistake as well)

More control over fastcgi_cache would be really nice(by this I mean options like auto clear cache when post are updated / added)

Overall W3 Total cache hasn't been receiving updates in a long time so I think it's a really good idea to remove it from EE.

[cm896]

Same result here, but didn't see a negative effect, pagespeed does not concat all JS or CSS to one request. I don't see it as a problem when looking at HAR files, or the network waterfall, the impact of calling more than one css or js group concatenated file doesn't seem significant. Also, the page scores aren't dinged incredibly for this behavior. I was never able to get the W3TC to minify correctly and had to use another plugin to do so, which was also hit or miss. I think because I have so many horribly progammed plugins, that don't always use the enqueue scripts command, late css and js inclusions mess with W3TC and the other plugins. I use pagespeed to defer JS, move CSS to head, and that helps the score and performance more than the minify, even though I am sending a huge amount of css and js on each page load.

[anandslab]

@xkidro Your can achieve fastcgi cache auto-clearing with Nginx Helper plugin from rtCamp. And my experience with pagespeed is the same in that minify does not work as well as W3TC (for me). No concatenation either. Not sure if I am doing something wrong.

If I can make pagespeed to do minify and CDN well. Then I would be replacing W3TC with only 2 lightweight plugins: Nginx Helper and Redis, which I would not mind.

[MiteshShah]

@xkidro and @htpcBeginner What issue did you face to minify using pagespeed.

i.e all the minify doesn't work or some of files cause the issue.

[ddelaey]

I also would not mind to get rid of the W3TC plugin. Currently I get a gtmetrix pagespeed score of 97%, But I'm still looking to further improve the yslow score of 89%. The biggest score gain for me came from enabling gzip compression via nginx. The less WP plugins, the better I think, less chance on conflicts between plugins, less dependencies on the vendors of those plugins, and less to configure via WP. For me it's important that pages/posts automatically get cached when they are created/modified, and not on first access. I only since recently experiment with pagespeed (since it's supported via ee). The increase in gtmetrix score so far is minimal, a little increase in the yslow score. Mainly using 'pagespeed RewriteLevel CoreFilters;' for the moment, plus a few others. Need to experiment further with the other filters. I wonder if the pagespeed directives in the browser html code have a positive influence on rankings ;-) I also use SSL, and indeed SPDY is a must, together with ssl_stapling. Getting an A+ quotation on SSL is IMO part of respecting site visitors.

Extensive documentation on all this would be highly recommend and appreciated.

Would be great if we can share/discuss the config files we use, to learn to get the maximum out of it :-).

[rahul286]

Wow. I am thrilled to see so much response on this. :-)

Let me add few more details which I hope will answer most queries.

Full Page Caching

--wpfc can do it all. Cache clearing can be handled via https://wordpress.org/plugins/nginx-helper/ plugin.

Database Cache

Not needed IMHO. MySQL query cache is there. WordPress object-cache is also there.

Also for every wordpress request there has to be atleast one mysql connection from wp-load.php so no database cache plugin can save 100% of mysql requests.

Still, I am no expert but as a user I stopped using this feature since long time and never ran into any problem.

Object Cache

object-cache.php is only file needed. It can be added by EE. Or can be shipped inside https://wordpress.org/plugins/nginx-helper/ plugin. In any case, it won't add a new plugin to plugin management screen.

Only issue we are facing is - choosing right object-cache.php. There are many for memcache and redis each. Please check #472 and #351

One that is working on our main site (rtcamp.com) is - https://wordpress.org/plugins/redis-cache/ , but it's not working for some sites.

For memcache this worked for many - https://github.com/tollmanz/wordpress-pecl-memcached-object-cache sites but couldn't get it working on our main site.

So basically we do not have a single object-cache.php working for all. There are some object-cache.php implementations which slow down things, instead of speeding them up!

As of now we are slightly inclined towards memcache because we have some nice nginx modules on list which makes use of memcache. And having memcache + redis on same server seems overkill. If you have any suggestion about memcache v/s redis - please add it to #472

CDN

CDN is very simple.

I just added a line to rtcamp.com's nginx config to get CDN working.

subs_filter https://rtcamp.com/wp-content/uploads/ https://rtcamp.r.worldssl.net/wp-content/uploads/;

Please note that above works for origin pull/mirror style CDN e.g. Amazon cloudfront. Not for push/storage style cdn e.g. Amazon S3.

All we need is CDN domain. I guess nginx-helper can have input field for CDN domain and write a config line like above in a predefined file e.g. /var/www/example.com/conf/nginx/cdn.conf. EE can monitor that file location and reload nginx config to get CDN working.

Combine + Minify

Be it pagespeed or w3tc or any solution - I could never get minify + combine working on all sites 100% of the time! I guess it has more to do with markup and actual css/js codes and inter-dependency. Because there are cases, where files got combined and minified but web pages broke in terms of functionality.

As of now pagespeed is creating an issue with full page cache - please check https://github.com/rtCamp/easyengine/issues/497

On sidenote. SSL site with SPDY don't need to worry about this much. I do not have data to backup this claim but I see few thing in SPDY which takes care of this in different way.

But in case of EE, even if it installs pagespeed or w3tc, can't configure minify + combine itself. So if we remove --w3tc and somebody need to use W3 Total Cache to minify it, they can still do so. We can leave minify related config in EE for sometime or move it to documentation.

I feel its important to remove options that don't add much value, so we can improve options that most people use.

Thanks again for all the feedback. I think we will be able to streamline this by June. :-)

[rahul286]

Static cache is high speed solution

Agree. That is why we recommend fastcgi-cache which also creates static files on disk.

and ideal for CDN

I did not get this part. WP Super Cache's static cache files are accesses internally and can not be used via CDN as far as I know.

WP with fastcgi + WP Super Cache

Not needed. Fastcgi cache is "full page" caching alternative to WP Super Cache as well W3 Total Cache.

[JosefJezek]

How to setup fastcgi cache with CDN for full page caching?

[rahul286]

@JosefJezek CDN is not used for full-page caching. It's used to serve images, css, js and other assets.

If you want entire site behind CDN, you can try something like http://cloudflare.com/

[JosefJezek]

Super cache and total cache generate html static files for CDN.

[rahul286]

Super cache and total cache generate html static files for CDN.

No. Those static files are not served via CDN. Web-server rules serve them internally.

See lines: https://github.com/rtCamp/easyengine/blob/v3.1.1/ee/cli/templates/wpsc.mustache#L23

and https://github.com/rtCamp/easyengine/blob/v3.1.1/ee/cli/templates/w3tc.mustache#L22

[JosefJezek]

But Super Cache and Total Cache support html static files on CDN. Static html files on CDN is best performance solution. We have ajax comments plugins and WP API for dynamic content.

Check out the link https://gist.github.com/JosefJezek/14c001a7a5000a2b0b08

[anandslab]

@JosefJezek Rahul is right. Super cache and W3TC create static files but those are stored locally. Not on CDN. So W3TC only pushes JS, CSS, and Images (and others) but not HTML to CDN.

[anandslab]

@MiteshShah For me Minify did not work at all. I even tried combine alone. That did not work either. I am on EE by the way.

[JosefJezek]

Now is time for new static themes based on WP API and PHP only for API. ;-)

https://gist.github.com/8ecd8bc35ce0e728d549

[JosefJezek]

Ok, then fastcgi cache is better solution.

[rahul286]

@JosefJezek The solution at link https://gist.github.com/JosefJezek/14c001a7a5000a2b0b08 is quite different.

Easiest way to have entire site on CDN is to use http://cloudflare.com/

I am interested in this area (putting entire site behind CDN) but it has many limitations like:

1. We need to access /wp-admin from different URL as most CDN won't handle dynamic part. Also wordpress cannot have buddypress, store, etc.
2. Putting static sites on a single FTP/S3 server only is not useful. It must be CDN to reduce end-user latency. Single static server latency is what we get anyway from current caching solutions (fastcgi, w3t, wpsuper cache).

Overall, the idea is nice but complicated as it needs few things on network/DNS level. So it's outside EE scope.

For any such sites, where I really don't need any dynamic part from user, I will prefer to go with static site generators. There are too many of them http://www.staticgen.com/ (my fav is hugo - http://gohugo.io/)

[JosefJezek]

Yes, I love metalsmith.io, but I need good backend for authors and orginal plugins about WP. My idea is generate static html pages/posts using WP and place to CDN. All dynamic content for frontend based on WP API.

WP have cool project maera.io as good base for themes.

PHP admin backend is only for admin, authors of articles, moderators.

We have WP API v1.2, now is time for change themes and plugins.

I am working on theme based on Polymer and WP API + Pods. ;-)

[rahul286]

@JosefJezek sounds like a nice project.

I like static site generators but writing process seems complicated for non tech-savvy users.

There is a discussion in hugo community to solve this problem in different way. You may like to read it - http://discuss.gohugo.io/t/web-based-editor/155/ :-)

Anyway, if you find a nice workaround, please share.

[MiteshShah]

+1

[MiteshShah]

@all Testing http://poll.gitrun.com/rtcamp/easyengine/issues/509 so ignore this +1

[tinbot-development]

Hi,

Interesting thread. I would like to see a conclusion or devinitive EE statement to end this discussion. What do recommend? Managed WP hosts do all caching without using any plugin. I want to offer this for my customers but this thread confuses me.

Would prefer to be able to do all caching via EE with no need for any plugin. Is this possible? Also, I use MAXCDN for all my sites, when the EE plugin system is released, will I be able to use the MAXCDN api to add all new sites to my CDN with out having to do any config?

http://wp-rocket.me/features/ looks good. I want to offer all over these services without ant plugin required, all automatically configured when I add a new site.

Thanks - you are doing a great job.

[andrewklimek]

Hi @rahul286 I am happy about the changes you've proposed. I only use --wpfc and I only use W#TC for object cache. I'd be more than happy if you pick a object cache as you described and have one fewer plugin. For minifying, I have also had trouble with w3tc and pagespeed, but Autoptimize has been the best for me: https://wordpress.org/plugins/autoptimize/

Thanks for ee and keep up the great momentum!

[janiosarmento]

If you drop support to a full page cache (because of FastCGI cache) I'll have to add support to WP Super Cache by myself: several customers of mine have low RAM systems (1GB), but their blogs have a very large number of pages, making --wpfc unfeasible for them.

[doliver3]

In regards to "Full page cache (Nginx fastcgi_cache is replacement)", the nginx install must be easier for people in my opinion if you are to replace supercache and W3TC which have cache clearing capabilities when editing a post or page without requiring a rebuild/reinstall of nginx. Having to compile nginx with a special third-party plugin is a big burden and downside of using the nginx helper plugin.

it is nice that you have a ppa to install nginx without compiling yourself but it really doesn't meet our needs. First, I have no idea which nginx configure compile options you included your ppa and second, I want to be able to choose which compile options to use. If you look at the Ubuntu install packages for nginx, they have several - light, full, extras, etc. I would much prefer to use one of the standard Ubuntu packages for nginx. I don't really know why it requires a third party nginx plugin just to clear the nginx fastcgi cache on demand.

[rahul286]

@doliver3 EasyEngine ships with nginx build which has all modules pre-complied. Also, nginx-helper is already gets installed for nginx-fastcgi-cache (--wpfc) and redis-fullpage-cache (--wpredis). Apart from that nginx-helper is configured to purge relevant pages fro cache when a post/page/CPT is published/edited/deleted.

So basically for EasyEngine users there is no difference in configuring any 4 type of full-page cache in terms of human efforts.

[kymcarlsson]

@rahul286 glad to see Redis made it in. +1 for a Redis object cache only option (that can be used with standard battle tested nginx full page cache option) or better yet only use memcached for w3total cache option and otherwise use redis as the default object cache .

[s-a-s-k-i-a]

I'd love to try out redis on my 100% SSL website. But I read that Redis does not support ssl. Is that true?

If I set up a WP site with ee command site and --w3tc, and have it all set up, how can I move my site to redis in combination with pagespeed - dropping w3tc in total without screwing up?

Thanks for the info!

If SSL works perfectly with redis, I am in for redis object cache with ee wp sites.

[robrecord]

+1 for nginx-fastcgi-cache and redis-object-cache together; removal of w3tc.

[christoferw]

+1 for removal of w3tc.

[janiosarmento]

+1 for removal of w3tc and memcached.

[ichpen]

+1 removal of w3tc, no need to have it

[cm896]

+1 removal of w3tc and memcached

[vitaligent]

+1 for removal of w3tc and memcached. W3TC especially never felt right being in this project.

[RavanH]

+1 for drop of w3tc support, it can be moved to the documentation.

Can anyone provide a (link to a) quick draft on how to remove W3TC and all related EE nginx rules safely? I'd like to test object-cache.php from https://wordpress.org/plugins/memcached/ but want W3TC out of the way first...

[rsmith4321]

Just update your cache type like if creating a new site https://easyengine.io/docs/commands/site/update/

On Sat, Mar 19, 2016 at 6:30 PM, RavanH notifications@github.com wrote:

+1 for drop of w3tc support, it can be moved to the documentation.

Can anyone provide a (link to a) quick draft on how to remove W3TC and all related EE nginx rules safely? I'd like to test object-cache.php from https://wordpress.org/plugins/memcached/ but want W3TC out of the way first...

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/EasyEngine/easyengine/issues/509#issuecomment-198800297

[rustyguts]

+1 for nginx-fastcgi-cache and redis-object-cache together; removal of w3tc.

[rahul286]

Update We decided to keep only one caching backend in v4. Please refer #1015