503 Guru Meditation | ESI processing not enabled | Dummy Block not being processed

[jasondsilva]

Hello, I am looking at trying to use varnish in a shared environment.

I have turpentine installed and without applying the vcl, pages are being served via Varnish. However, the moment I select "Apply VCL On Config Change" in Varnish Options, my domain goes down and I get a 503 Guru Meditation error. This also affects other domains on the server. However, if I restart Varnish, all the domains start working fine.

[jakelly]

@jasondsilva:

If I understand correctly, I just resolved this same issue yesterday!

Configuration:

• I have Centos running Varnish on VMWare running from my laptop.
• My laptop is hosting the Magento website
• Varnish on Centos points back to the Magento site as the backend
• When applying changes I received 503 Guru Meditation error

The problem that I discovered, is that the firewall on Centos was preventing communication on the admin port (6082). Turning off the firewall resolved my issue. That’s the short version. I plan to document it on my blog sometime today. If you’d like, I can reply back with a link to my write up?

[jasondsilva]

Hi @jakelly , I have been trying this on a shared server. Did not try it on CentOS. Apparently, Turpentine generates the vcl file and directly writes it to the port specified in the magento backend.

As such, if its a shared server used by multiple domains using the same port for Varnish, all the domains get affected. I am currently looking into https://github.com/nexcess/magento-turpentine/issues/174 to see if this solves the issue for me. However, the above issue creator has not responded as yet whether it worked for him.

Will post an update on this if I make any progress.

[jasondsilva]

Apparently, unixy issue. Currently I have an nginx server and varnish 2.1 installed. Now my server does not crash.

However, I get "ESI processing not enabled" for the header and the footer. Weirdly, the response headers from the server states that esi processing is enabled.

Response headers from the server: Age:501 Connection:keep-alive Content-Encoding:gzip Content-Length:12544 Content-Type:text/html; charset=UTF-8 Date:Wed, 08 Jan 2014 15:23:04 GMT Server:nginx Via:1.1 varnish X-Powered-By:PHP/5.3.27 X-Turpentine-Cache:1 X-Turpentine-Esi:1 X-Varnish:1002475301 1002475199 X-Varnish-Hits:1 X-Varnish-Set-Cookie:frontend=000000004a164a168a270554169c39e2; expires=Wed, 08-Jan-2014 16:14:42 GMT; path=/; domain=sampledomain.com; httponly

[j0ney3]

@jasondsilva I had a TON of issues with Unixy Varnish, I think this was due to it's loads of vcl files that it includes, I could never merge them correctly w/ the Turpentine generated vcl. I eventually abandoned Unixy and am just running a regular Varnish install on CentOS. I use Turpentine to generate a default.vcl to a dummy location, then customize it to ignore the vhosts that I don't want to mess with, by doing the following in default.vcl, this basically bypasses Varnish for the domains that I don't want to mess with:

#in sub vcl_recv, add conditions for all of your defined backends, I define one per vhost
 if (req.http.host ~ "(www\.|js\.|skin\.)?varnishedmagentoinstall\.com") {
        set req.backend = magento;
        #return(pass);
    } else if (req.http.host ~ "(www\.)?otherdomain\.com") {
        set req.backend = other;
        return(pass);
    } else if (req.http.host ~ "(www\.|js\.|skin\.)?another\.com") {
        set req.backend = another;
    } else if (req.http.host ~ "(www\.)?andanother\.com") {
        set req.backend = andanother;
        return(lookup);
    } else if (req.http.host ~ "(www\.)?lastone\.com") {
        set req.backend = last;
        return(lookup);
    } else {
        return(pass);
    }

[jasondsilva]

Hi, @j0ney3 . Appreciate the help. My current install has only 1 domain though. I am trying to get the entire setup working for 1 domain and then expand to other domains. The initial unixy setup had multiple domains, which I have now scrapped. I am still getting the "ESI processing not enabled" though. Would appreciate some comments/opinions on solving that.

https://gist.github.com/jasondsilva/8329105 - Link to default.vcl that is generated by turpentine which is being used by the server.

https://gist.github.com/jasondsilva/8329486 - Link to Turpentine log generated upon flushing cache. Notice that the header and footer do not have any "ESI injection" comments attached to them.

https://gist.github.com/jasondsilva/8330112 - Link to Varnish log generated upon opening up the home page of the site.

https://github.com/nexcess/magento-turpentine/issues/37 - Checked this out but to no avail. I do not have mod_pagespeed for nginx.

[j0ney3]

@jasondsilva Ok, so I'm no expert, but you should probably have varnish admin on a separate port than your other backends. Not sure about this after looking at the Turpentine default configuration, anyone know?

What version of Varnish are you running?

And this is a stupid question, but I've done it, lol, so I gotta ask, you definitely have the Varnish related cache settings enabled in System->Cache Management right?

This is what it should look like if the header block is being detected and injected in the system.log:

2014-01-09T14:30:59+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: header
2014-01-09T14:30:59+00:00 INFO (6): TURPENTINE: Block check passed, injecting block: header

So your block is definitely being detected properly, and the default turpentine_esi.xml should pick that up and inject it. Do you have any overrides in your template/theme, maybe local.xml or another version of turpentine_esi.xml?

EDIT: Forgot to ask, what happens when you hit your ESI url for the header directly?

[jasondsilva]

@j0ney3 Apologies for the late reply. I am using Varnish 2.1 and Magento 1.7 CE. Yes, I have cache setting enabled for Varnish blocks and pages.

In the turpentine generated VCL file the following needed to be added in vcl_recv to enable ESI processing, unset req.http.Accept-Encoding; unset req.http.Vary;

After adding this, things started working. However, dummy blocks are not getting processed.

https://gist.github.com/jasondsilva/ff8c9340606f4a9f13ae

Above is the modification I have done to the turpentine_esi.xml file for the dummy block. In my head.phtml file I have added to following, <?php echo $this->getLayout()->getBlock('head')->getTitle(); ?>

However, after restarting varnish, I get the following error in the nginx log,

2014/01/17 17:36:28 [error] 11951#0: *986 FastCGI sent in stderr: "PHP message: PHP Fatal error: Call to a member function getTitle() on a non-object in /home/accountname/public_html/app/design/frontend/base/default/template/page/html/header.phtml on line 6" while reading response header from upstream, client: 10.0.0.200, server: www.demosite.com, request: "GET /turpentine/esi/getBlock/method/esi/access/private/ttl/3600/hmac/b492c2baea3266d9ee63e5464e1ebdb3bab520f02228192623dfcc444e80b69f/data/6kDoL0D.sxu1tzLRl.0q4VfmehBUHGDcEkk7bHAFtChMyzxqWkQucmhVojRXQH1RVXmYq.RFgnyEgXBxS3h5v7Ck1DCsmcLZFnK3aycVuXJXC4v5nK62AVok5c6JSZZ7PV-zhRn6kSqz10ardcKFY2REGAMvXoxEIt.bwVPyuxxf4G-aQ49pA60290PuDNwYfukrMZp1LTvT1HWXBa7G6zkceeiCo5bvS6RsbOJHzxzG93zmO3QnS7uWSjyxJx-F9FkbmwE1nEQ62ev3E5CUo55YPpBGGxeK40Ra43Rps.2bpE1DhUI68KsPieZfptvYPW5xBEHnnG00Q3rCLt5hW.9YcDLz65Ke1mHRZq7xaMef0vEIPSUI3uZIZ45R7Rk2WptH4TLLWYyrE9J3eJtMD0Q7uRRjanpyqSIDu2FI.xU=/ HTTP/1.1", upstream: "fastcgi://unix:/opt/pifpm/fpmsockets/accountname.sock:", host: "demosite.com"

It looks like turpentine is not processing the dummy blocks. However, if any block does not have a dependency, it works fine.

[jasondsilva]

Issue still persists, magento varnish log gives the following,

2014-01-17T12:51:51+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: head 2014-01-17T12:51:51+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: js_cookies 2014-01-17T12:51:51+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: optional_zip_countries 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: after_body_start 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: google_analytics 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: turpentine_notices 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: global_notices 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: header 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Block check passed, injecting block: header 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: breadcrumbs 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: global_messages 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Block check passed, injecting block: global_messages 2014-01-17T12:51:52+00:00 INFO (6): TURPENTINE: Checking ESI block candidate: content

As you can see, the dummy block has been added in the xml file (https://gist.github.com/jasondsilva/ff8c9340606f4a9f13ae), however, magento's varnishlog does not show any entry for head block being processed while the header block is being processed.

@aheadley Would appreciate if you could drop some pointers. Apologies for tagging you out of the blue.

[j0ney3]

So it looks like the header block is being injected properly now. You should be able to switch out your getTitle() call in head.phtml w/

$this->getTitle();

Is there something in your head.phtml that needs to be injected on a per-user basis? Only asking b/c mine can be cached globally. What happens if you strip out the

<dummy_blocks>
    <head/>
</dummy_blocks>

[jasondsilva]

@j0ney3 Thanks for the response. I need the call to head-getTitle on the header block to duplicate the title of the page.

Removing the dummy block from the turpentine xml does get the site to work. Is there some additional param that I need to pass to call the head block inside the header block?

[j0ney3]

Oh I see, you're using that on header.phtml, not head.phtml, my misunderstanding, sorry.
So with that dummy block removed, you're not seeing any errors OR output from the call?

<?php echo $this->getLayout()->getBlock('head')->getTitle(); ?>

[jasondsilva]

@j0ney3 apologies. I should have been clearer. If I remove the dummy block AND the reference to gettitle, the site works fine. If I remove the dummy block AND keep the reference to gettitle, the header breaks. If I keep the dummy block and the reference to gettitle the site breaks.

[j0ney3]

Yeah, I get those exact same results when I try...I guess you could add the current page title to the registry and grab it from there, I know it's shitty but at least it would work for now

[jasondsilva]

@j0ney3 yeah. That would probably solve the title issue. However if I create any custom module which has above said dependencies or similar, turpentine would fail to render.

Unfortunately, I cannot seem to find references where anyone has solved this.

[jasondsilva]

@j0ney3 @jakelly Hi guys. I still have not been able to make any headway in this. The dummy block processing apparently seems to be only limited to javascript calls to functions that are present in the dummy block. As such, I cannot call php functions related to the dummy block.

Can this be confirmed?

[j0ney3]

Yep, I couldn't get at any of the functions related to 'head' without a fatal on my end. It really doesn't make much sense to me as the ESI request SHOULD have access to $this->getLayout(). There's another bug on here that sounds very similar: https://github.com/nexcess/magento-turpentine/issues/178