W3 Total Cache is not compatible with NGINX (yet), so this test is not really saying that much 🙂 Using APC doesn’t actually speed up loading on all sites either. Also, what were you other W3TC settings? Did you use DB caching at all?
I did use custom nginx re-write rules for both W3TC and WP-SC to serve up static HTML if available (when W3TC is using the disk for page cache, obv). The rewrite rules gave both plug-ins a good jump in req/sec which are the numbers shown. Without, the numbers aren’t much better than no caching due to nginx having nothing static to serve and going to PHP to get content. I also tested HyperCache, DB Cache reloaded, and Ask Apache Crazy Cache. None of these were much better than no cache, again due to needing Nginx rewrite rules.
APC was used for DB caching in the second W3TC tests.
With the W3TC APC tests, I have minification on via APC, line break & comment removal on for HTML and inline styles and CSS.
There are some misconceptions about how to compare caching plugins that seem to be impossible to dispel.
The most important of which is that W3TC can only be compared to other page caching plugins when ONLY page caching is used.
When comparing any plugin only the same methods should be compared, in other words, the minify cache, database cache and CDN functionality should all be disabled. WPSC vs W3TC requires full on (super cache) mode to be compared to disk enhanced mode of W3TC. Since nginx is not supported, I cannot feel certain that the directives used are ideal/optimized.
When a caching plugin that uses PHP (instead of HTML files on disk) is compared to another, you’re comparing the efficiency of the plugin code and the throughput of the storage method used (disk or memory). In the case of W3TC the goal for the disk basic was to allow users to quickly implement reliable caching without any need for .htaccess directives, but the scheme used is “intense” for slow disks, depending on your settings.
Caching pages to memory still invokes PHP and therefore (depending on a number of factors) is not faster than instead caching to static html files.
For a strict page caching test, all one must do is enable page caching (disable the others), prime the cache and then run their benchmark software. What they will be testing once the cache is primed is the performance of their chosen web server (if the cache is using static html files). So in effect, any caching plugin that creates html files should be comparable and only differ based on the complexity of their directives for disk caching. So all of these kinds of tests actually are not helpful for non-techies because it’s never explained what the tests really mean. Furthermore, I saw no indication of whether or not gzip compression was used, which contributes to higher requests per second.
I think the link yoast refers to is more useful reference for those concerned about these types of tests because it has a clear method that can be duplicated and accurate results. There’s a lot more to say on this topic, but the posts I see popping around sites like tutorial9 and others I don’t feel actually help anyone. Individual mileage varies and sites/servers do also. W3TC allows you to optimize WordPress for your site/server/goals, none of the others do or can.
rograndom
Thank you Frederick. And yes, gzip compression is on in W3TC, you can see that the HTML transferred for W3TC is lower than the others.
I run many WordPress sites that generate 100’s to 10,000’s page views a day. Each hit to PHP/MySQL to render the page is a bottleneck. My goal of this comparison is to find the caching plugin that can help serve up as many pages per day/hour/minute as possible and keep the system load as low as possible. Once W3TC supports Nginx better I would love to get my hands on it. Or, alternately, if there is a W3TC setup you can suggest that would be awesome.
I would be happy to give you a couple tips, but to do that I’d need to ask you a few questions about your site which we can do offline. You have my email address now I think.
Meanwhile it would be nice if you could add disk enhanced test to the line items above and also add version numbers for each of the plugins. I really hope users don’t even try wp cache at this point, it’s really not meant to be used anymore, even Donncha will indicate the same.
W3 Total Cache is not compatible with NGINX (yet), so this test is not really saying that much 🙂 Using APC doesn’t actually speed up loading on all sites either. Also, what were you other W3TC settings? Did you use DB caching at all?
This might be nice to read: http://cd34.com/blog/scalability/wordpress-cache-plugin-benchmarks/, it’s a pretty thorough comparison of caching plugins.
I did use custom nginx re-write rules for both W3TC and WP-SC to serve up static HTML if available (when W3TC is using the disk for page cache, obv). The rewrite rules gave both plug-ins a good jump in req/sec which are the numbers shown. Without, the numbers aren’t much better than no caching due to nginx having nothing static to serve and going to PHP to get content. I also tested HyperCache, DB Cache reloaded, and Ask Apache Crazy Cache. None of these were much better than no cache, again due to needing Nginx rewrite rules.
APC was used for DB caching in the second W3TC tests.
With the W3TC APC tests, I have minification on via APC, line break & comment removal on for HTML and inline styles and CSS.
Also, no CDN settings were used.
There are some misconceptions about how to compare caching plugins that seem to be impossible to dispel.
The most important of which is that W3TC can only be compared to other page caching plugins when ONLY page caching is used.
When comparing any plugin only the same methods should be compared, in other words, the minify cache, database cache and CDN functionality should all be disabled. WPSC vs W3TC requires full on (super cache) mode to be compared to disk enhanced mode of W3TC. Since nginx is not supported, I cannot feel certain that the directives used are ideal/optimized.
When a caching plugin that uses PHP (instead of HTML files on disk) is compared to another, you’re comparing the efficiency of the plugin code and the throughput of the storage method used (disk or memory). In the case of W3TC the goal for the disk basic was to allow users to quickly implement reliable caching without any need for .htaccess directives, but the scheme used is “intense” for slow disks, depending on your settings.
Caching pages to memory still invokes PHP and therefore (depending on a number of factors) is not faster than instead caching to static html files.
For a strict page caching test, all one must do is enable page caching (disable the others), prime the cache and then run their benchmark software. What they will be testing once the cache is primed is the performance of their chosen web server (if the cache is using static html files). So in effect, any caching plugin that creates html files should be comparable and only differ based on the complexity of their directives for disk caching. So all of these kinds of tests actually are not helpful for non-techies because it’s never explained what the tests really mean. Furthermore, I saw no indication of whether or not gzip compression was used, which contributes to higher requests per second.
I think the link yoast refers to is more useful reference for those concerned about these types of tests because it has a clear method that can be duplicated and accurate results. There’s a lot more to say on this topic, but the posts I see popping around sites like tutorial9 and others I don’t feel actually help anyone. Individual mileage varies and sites/servers do also. W3TC allows you to optimize WordPress for your site/server/goals, none of the others do or can.
Thank you Frederick. And yes, gzip compression is on in W3TC, you can see that the HTML transferred for W3TC is lower than the others.
I run many WordPress sites that generate 100’s to 10,000’s page views a day. Each hit to PHP/MySQL to render the page is a bottleneck. My goal of this comparison is to find the caching plugin that can help serve up as many pages per day/hour/minute as possible and keep the system load as low as possible. Once W3TC supports Nginx better I would love to get my hands on it. Or, alternately, if there is a W3TC setup you can suggest that would be awesome.
I would be happy to give you a couple tips, but to do that I’d need to ask you a few questions about your site which we can do offline. You have my email address now I think.
Meanwhile it would be nice if you could add disk enhanced test to the line items above and also add version numbers for each of the plugins. I really hope users don’t even try wp cache at this point, it’s really not meant to be used anymore, even Donncha will indicate the same.