not so big difference: could it be my settings?

[x86fantini]

Hi first let me thank you for your effort to mantain this extension free/opensource.

I have a test environment http://37.247.54.101/ with magento 1.7.0.2 fresh and clean install.

environment 1 (test was run with 30 users for 45 seconds) magespeedtest.com : 11 trans/sec - 1.5 sec each debian 7 nginx php5-fpm apc redis modified local.xml

environment 2 (test was run with 30 users for 45 seconds) magespeedtest.com : 20 trans/sec - 0.8 sec each debian 7 nginx php5-fpm apc redis varnish 3.0 modified local.xml turpentine varnish cache

it seems not so much difference. can it be possibile? this is one of my first varnish installation, maybe it's me.

can i paste here some of my config files? what do you prefer..nginx, varnish..localxml..

thank you for your time Simone

[aheadley]

It looks like you're testing with MageSpeedTest? Based on the results you're probably running into latency from the US West site + the low concurrency limit for free users. For instance, compare our sip400-demo site with 20 users and 100 users.

In other words, it's not an issue with your site's speed, just a limit of what the benchmark tool can show you.

[x86fantini]

ok, so mines are 20 trans/sec while u get 36, almost double. it can be latency, but i do not think "that" much.

can varnish daemon configuration (or missconfiguration) influence on the speed test? mine it's like this right now:

DAEMON_OPTS="-a :80 \ -T 127.0.0.1:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -p cli_buffer=16384 -p thread_pools=2 -p thread_pool_min=100 -p thread_pool_max=1000 -p thread_pool_add_delay=2 -p cli_timeout=20 -p session_linger=100 -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,2G"

since i'm testing this on a 1gb ram vps (on purpose) i can not allocate varnish to run in "smalloc"

any other suggesting to better tune my system?

thx Simone

[aheadley]

Everything I've read about "tuning" Varnish basically said to not do it, the defaults are fine for 99% of sites. I really think the latency is the issue since your site looks to be hosted in Italy. Can you look in your webserver logs and find the IP of the MageSpeedTest server that did the test and try pinging it from your server to see what kind of latency you're getting?

[x86fantini]

ok, reverted to stoc varnish config, and allocated memory usage (-s malloc,256m) ping to magespeed server it's not responding, but traceing shows 165ms delay

traceroute 204.236.135.202
traceroute to 204.236.135.202 (204.236.135.202), 30 hops max, 60 byte packets
 1  pm31.prometeus.net (37.247.54.5)  0.050 ms  0.017 ms  0.014 ms
 2  gw-cdlan-2.prometeus.cdlan.net (217.171.46.253)  0.411 ms  0.510 ms  0.602 ms
 3  212.73.241.129 (212.73.241.129)  0.611 ms  0.612 ms  0.598 ms
 4  ae-14-14.ebr1.Frankfurt1.Level3.net (4.69.142.194)  155.379 ms  155.430 ms  155.375 ms
 5  * * ae-71-71.csw2.Frankfurt1.Level3.net (4.69.140.6)  155.269 ms
 6  ae-82-82.ebr2.Frankfurt1.Level3.net (4.69.140.25)  155.065 ms ae-72-72.ebr2.Frankfurt1.Level3.net (4.69.140.21)  155.450 ms  155.396 ms
 7  ae-24-24.ebr2.London1.Level3.net (4.69.148.197)  155.376 ms  155.449 ms ae-21-21.ebr2.London1.Level3.net (4.69.148.185)  155.614 ms
 8  ae-41-41.ebr1.NewYork1.Level3.net (4.69.137.66)  155.100 ms ae-44-44.ebr1.NewYork1.Level3.net (4.69.137.78)  155.061 ms ae-41-41.ebr1.NewYork1.Level3.net (4.69.137.66)  155.044 ms
 9  ae-91-91.csw4.NewYork1.Level3.net (4.69.134.78)  155.255 ms  155.565 ms ae-71-71.csw2.NewYork1.Level3.net (4.69.134.70)  155.040 ms
10  ae-62-62.ebr2.NewYork1.Level3.net (4.69.148.33)  155.375 ms ae-72-72.ebr2.NewYork1.Level3.net (4.69.148.37)  155.374 ms  155.441 ms
11  4.69.135.185 (4.69.135.185)  154.885 ms  155.005 ms  154.812 ms
12  ae-71-71.csw2.SanJose1.Level3.net (4.69.153.6)  163.422 ms  155.092 ms ae-81-81.csw3.SanJose1.Level3.net (4.69.153.10)  155.239 ms
13  ae-42-80.car2.SanJose2.Level3.net (4.69.152.140)  155.051 ms ae-12-70.car2.SanJose2.Level3.net (4.69.152.76)  155.139 ms ae-32-90.car2.SanJose2.Level3.net (4.69.152.204)  155.241 ms
14  AMAZON.COM.car2.SanJose2.Level3.net (4.59.4.10)  160.321 ms  160.586 ms  160.581 ms
15  205.251.229.46 (205.251.229.46)  163.149 ms  163.099 ms  163.049 ms
16  72.21.222.19 (72.21.222.19)  161.130 ms  161.287 ms  161.162 ms
17  216.182.236.109 (216.182.236.109)  164.715 ms 216.182.236.103 (216.182.236.103)  165.100 ms 216.182.236.109 (216.182.236.109)  164.800 ms
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *^C

not so bad i gues. but now magespeedtest.com it's down (error 500) so i'll wait and then run again a speed test

thank you Simone

[x86fantini]

reverting back to stock varnish settings: it's even slower :(

http://www.magespeedtest.com/results?key=f467b4f1-6fb1-49a4-8340-8856cd56940d

see? now, with varnish on it's just like without it..

this is header respons:

Accept-Ranges bytes Age 0 Cache-Control max-age=86400 Connection keep-alive Content-Encoding gzip Content-Length 4518 Content-Type text/html; charset=UTF-8 Date Tue, 14 May 2013 19:45:10 GMT Expires Wed, 15 May 2013 19:45:10 GMT Powered-By www.x86host.it Pragma no-cache Server nginx Set-Cookie frontend=dha236gvrvh3u7cds7gai3k8s2; expires=Tue, 14-May-2013 20:45:10 GMT; path=/; domain=37.247.54.101; httponly Vary Accept-Encoding Via 1.1 varnish X-Powered-By PHP/5.4.15-1~dotdeb.2 X-Varnish 1632232684 Intestazioni di richiesta Accept text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 Accept-Encoding gzip, deflate Accept-Language it-IT,it;q=0.8,en-US;q=0.5,en;q=0.3 Cache-Control max-age=0 Connection keep-alive Cookie frontend=dha236gvrvh3u7cds7gai3k8s2 Host 37.247.54.101 Referer http://37.247.54.101/electronics/cell-phones.html User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:20.0) Gecko/20100101 Firefox/20.0

[csdougliss]

Out of curiosity I've just come across magespeedtest.com - for how many seconds did you run the test for?

[x86fantini]

i've had the free account, so 45 second and 30 users now i have the premium account, will run several tests and post it.

one thing i know isa that NOT always (as it seems for me) varnish is hitting, i sometime need to do a full cicle of remove cache, refresh cache, restart varnish/nginx/php5-fpm..before getting any good results

does anybody utilize varnish + nginx and can help me with a good nginx reverse proxy config?

thank you Simone

[x86fantini]

ok just run a test over my url, with magespeedtest.com

results (very poor) : http://www.magespeedtest.com/results?key=9dc0eb5c-0338-4f8f-aa53-f7b2434fd64a

here are headers

37.247.54.101 HTTP/1.1 200 OK Server: nginx Content-Type: text/html; charset=UTF-8 Vary: Accept-Encoding X-Powered-By: PHP/5.4.15-1~dotdeb.2 Set-Cookie: frontend=v8d6ll2jo5nao5l88vb7oln1h2; expires=Wed, 15-May-2013 12:25:03 GMT; path=/; domain=37.247.54.101; HttpOnly Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Powered-By: www.x86host.it Content-Encoding: gzip Content-Length: 4497 Accept-Ranges: bytes Date: Wed, 15 May 2013 11:25:03 GMT X-Varnish: 852070865 Age: 0 Via: 1.1 varnish Connection: keep-alive

http://www.isvarnishworking.com/ is sayng:

Varnish appears to be responding at that url, but the "Age" header is less than 1.

This means that either, 1) you checked right when Varnish cleared it's cache for that url, or 2) for whatever reason Varnish is not actually serving the content for that url from cache.

If it is the former, just check again and see if you get a more solid confirmation.

If it is the latter, perhaps one of the following is happening:

That url is excluded from the cache on purpose in the Varnish vcl file (in which case, yay! It's working.)
The application is sending cache headers that are telling Varnish not to serve that content from cache. This means you'll have to fix the cache headers the application is sending to Varnish. A lot of the time those headers are Cache-Control and/or Expires.
The application is setting a session cookie, which can prevent Varnish from serving content from cache. This means you'll have to update the application and make it not send a session cookie for anonymous traffic.
Internet Ghosts.

please help :)

[aheadley]

It looks like Varnish is running with the default VCL (which doesn't cache anything from Magento) instead of the Turpentine-generated VCL, you need to hit the Apply Varnish Config button in the Magento admin under System > Cache Management after restarting Varnish if you don't replace Varnish's startup VCL (/etc/varnish/default.vcl) with the Turpentine-generated VCL.

EDIT: Also, just so you know, once Turpentine has applied its VCL, unless Turpentine's debugging is turned on the isvarnishworking.com site won't give accurate results as Varnish will remove the "fingerprints" that that site is looking for.

[x86fantini]

debug it's alwasy on :) so, when i hit save settings, i just "save" the new vcl, but it does not apply on the fly the vcl to the deamon? Becouse each time i get:

    VCL successfully applied to: 127.0.0.1:6082
    The VCL file has been saved.
    The configuration has been saved.

should i press also "apply varnish config" under cache management?

thank you 4 ur time

[aheadley]

so, when i hit save settings, i just "save" the new vcl, but it does not apply on the fly the vcl to the deamon?

If you hit the Save Settings button while under System > Configuration on either the Varnish Options or Caching Options tabs, then the VCL will be generated and saved to $PATH_TO_MAGENTO/var/default.vcl. If the Apply VCL On Config Change option is enabled (which it is by default), then hitting the Save Settings button will also apply the generated config to the running Varnish instance(s).

The Save Varnish Config and Apply Varnish Config buttons under System > Cache Management allow you to do the corresponding actions manually so you don't generally need to use them if you hit the Save Settings button.

[x86fantini]

well then in my case there is a "bug" that prevent from applyng vcl to running deamon...mybe

WOW!!!

http://www.magespeedtest.com/results?key=7f0054f6-ba0f-4a7f-b16b-c67d20e74ec4

i've copied /var/default.vcl --> /etc/varnish/default.vcl (made a bck copy of the original vcl) manually then restarted nginx & varnish and BOOM

[x86fantini]

ok, think i've found the problem: it's memcache! i have a local.xml with both fast and slow backend, 2 cache levels.

although i was flushing cache, "something" was still there...so i have enabled only APC as cache backend, flush cache, flush storage cache (most important), restared everything and reapplyed vcl.

..working like a charm

[aheadley]

Glad you got it working.