memcached down - Githubissues

TheRoSS commented 12 years ago

Hi If the memcached server is down the very strange output is generated and I have no idea to control it by the module. Expected behaviour is to pass request to proxy without cache requesting.

srcache version: 0.12rc5

Nginx conf:

upstream storage {
  server localhost:8081;
}

upstream memcached {
  server localhost:11211;
  keepalive 512;
}

server {
    # Get static content from storage (control center) via memcached caching
    location /storage {
        set $key $uri;
        srcache_fetch GET /memc key=$key;
        srcache_store PUT /memc key=$key;
        proxy_pass http://storage;
    }

    location = /memc {
        internal;
        set $memc_key $arg_key;
        set $memc_exptime 10;
        memc_pass memcached;
    }

    # redirect server error pages to the static page /50x.html
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   html;
    }

Request:

curl -v http://localhost:8080/1001/1000/1.png

If memcached is up I have my .phg picture

If memcached is down the result is:

> GET /1001/1000/1.png HTTP/1.1
> User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> Host: localhost:8080
> Accept: */*
>
<html>
<head>
<title>The page is temporarily unavailable</title>
<style>
body { font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body bgcolor="white" text="black">
<table width="100%" height="100%">
<tr>
<td align="center" valign="middle">
The page you are looking for is temporarily unavailable.<br/>
Please try again later.
</td>
</tr>
</table>
</body>
</html>
HTTP/1.1 200 OK
Server: nginx/1.0.4
Date: Tue, 06 Mar 2012 08:54:00 GMT
Content-Type: image/png
Connection: keep-alive
Last-Modified: Thu, 18 Nov 2010 14:37:40 GMT
ETag: "40000a6a-8ca-49554bb626c14"
Accept-Ranges: bytes
Content-Length: 2250

�PNG
......  picture data  ......
<html>
<head>
<title>The page is temporarily unavailable</title>
<style>
body { font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body bgcolor="white" text="black">
<table width="100%" height="100%">
<tr>
<td align="center" valign="middle">
The page you are looking for is temporarily unavailable.<br/>
Please try again later.
</td>
</tr>
</table>
</body>
</html>
* Connection #0 to host localhost left intact
* Closing connection #0

agentzh commented 12 years ago

On Tue, Mar 6, 2012 at 8:35 PM, agentzh agentzh@gmail.com wrote:

On Tue, Mar 6, 2012 at 7:08 PM, TheRoSS reply@reply.github.com wrote:

# redirect server error pages to the static page /50x.html error_page 500 502 503 504 /50x.html; location = /50x.html { root html; }

The error_page directive will be inherited automatically by your /memc internal location. You should avoid that.

This limitation has been fixed in later versions of ngx_srcache. I suggest you upgrade ngx_srcache to the latest v0.13rc3 release. Thank you :)

Best, -agentzh

TheRoSS commented 12 years ago

It works. Thank you. If memcached is down I have a correct error page.

But I need something different. If request to get content from cache is failed because of dead memcached server, it should be redirected to proxy as if the cache miss occured. After content from proxy was received I wanna try to store it in cache as usually. The fail of this attempt should be ignored.

Is there any simple way to do it?

agentzh commented 12 years ago

On Wed, Mar 7, 2012 at 3:51 PM, TheRoSS reply@reply.github.com wrote:

It works. Thank you. If memcached is down I have a correct error page.

But I need something different. If request to get content from cache is failed because of dead memcached server, it should be redirected to proxy as if the cache miss occured. After content from proxy was received I wanna try to store it in cache as usually. The fail of this attempt should be ignored.

What you want is the expected behavior. It returns the error page of the failed memcached query? Are you using the latest ngx_srcache module?

Best, -agentzh

agentzh commented 12 years ago

I've just tried the following minimized example with the latest ngx_srcache, v0.13rc3:

error_page   500 502 503 504  /50x.html;
location = /50x.html {
   root   html;
}

location = /foo {
    srcache_fetch GET /memc $uri;
    srcache_store PUT /memc $uri;

    proxy_pass http://127.0.0.1:$server_port/hello;
}

location = /hello {
    echo hello;
    default_type text/css;
}

memc_connect_timeout 1ms;
location = /memc {
    internal;

    set $memc_key $query_string;
    set $memc_exptime 300;
    memc_pass www.google.com:1234;
}

Here, connecting to www.google.com:1234 must time out. And the accessing /foo achives the expected results:

$ curl -i localhost:8080/foo
HTTP/1.1 200 OK
Server: nginx/1.0.10
Date: Wed, 07 Mar 2012 09:01:34 GMT
Content-Type: text/css
Connection: keep-alive
Content-Length: 6

hello

Can you try this out on your side? Or can you give me a minimized example and detailed instructions for reproducing the problem?

Best, -agentzh

TheRoSS commented 12 years ago

Yes. All works as I expected. Thank you. I think there was a some network error while I tested srcache after upgrading it to last version, so backend was unreachable and I got 'page unavailable' error message. Today I did clean start and all worked nice on nginx 1.0.13 and nginx 1.0.4.

agentzh commented 12 years ago

Can you reproduce this issue now?

TheRoSS commented 12 years ago

No. Now it's all right. The issue can be closed.

openresty / srcache-nginx-module

memcached down #13