openresty / srcache-nginx-module

Transparent subrequest-based caching layout for arbitrary nginx locations.
http://wiki.nginx.org/NginxHttpSRCacheModule
474 stars 103 forks source link

How do you use redis with password? #60

Open onnimonni opened 7 years ago

onnimonni commented 7 years ago

Hey!

I would really like to use this module but the redis in my environment needs to have authentication turned on.

I tried to solve this by using content_by_lua but don't know how to.

This is what I would want to do but nginx redis or redis2 modules don't support redis passwords:

location /api {
     default_type text/css;

     set $key $uri;
     set_escape_uri $escaped_key $key;

     srcache_fetch GET /redis $key;
     srcache_store PUT /redis2 key=$escaped_key&exptime=120;

     # fastcgi_pass/proxy_pass/drizzle_pass/postgres_pass/echo/etc
 }

 location = /redis {
     internal;

     set_md5 $redis_key $args;
     redis_pass 127.0.0.1:6379;
 }

 location = /redis2 {
     internal;

     set_unescape_uri $exptime $arg_exptime;
     set_unescape_uri $key $arg_key;
     set_md5 $key;

     redis2_query set $key $echo_request_body;
     redis2_query expire $key $exptime;
     redis2_pass 127.0.0.1:6379;
 }
agentzh commented 7 years ago

@onnimonni Both ngx_redis and ngx_redis2 should support passwords. Please read their official documentation more carefully.

onnimonni commented 7 years ago

Thanks for your help!

I think I have read through them a hundred times already, but what I missed was the following redis command: http://redis.io/commands/AUTH.

I will try this soon but does the authentication work like this?

location = /redis2 {
     internal;

     set_unescape_uri $exptime $arg_exptime;
     set_unescape_uri $key $arg_key;
     set_md5 $key;

     redis2_query auth $password; <----- Use the password here
     redis2_query set $key $echo_request_body;
     redis2_query expire $key $exptime;
     redis2_pass 127.0.0.1:6379;
 }

I think I missed the following one?

redis2_query Multiple instances of this directive are allowed in a single location and these queries will be pipelined. https://github.com/openresty/redis2-nginx-module#redis2_query

Side topic: If I use this with docker container enviroment variables is the preferred way to read the configuration like this:

set_by_lua $redis_host 'os.getenv("REDIS_HOST")';
set_by_lua $redis_pass 'os.getenv("REDIS_PASSWORD")';
set_by_lua $redis_port 'os.getenv("REDIS_PORT")';

Is there better ways than this?

Thanks in advance! Thanks for creating such useful tools :)!

onnimonni commented 7 years ago

And I did also read the ngx_redis readme: https://www.nginx.com/resources/wiki/modules/redis/#variables

And I can't really figure it out.

There's only mention of these two variables:

$redis_key - The value of the redis key.

$redis_db - The number of redis database (required for < 0.3.4).

But no mentioning about pipelines or arguments or anyway to provide the AUTH 😞.

onnimonni commented 7 years ago

I managed to do this finally using this fork of the redis module: https://github.com/Yongke/ngx_http_redis-0.3.7

agentzh commented 7 years ago

@onnimonni Oh, I didn't know the official ngx_redis module does not support auth variables. Will you ask the original author of ngx_redis to add support for it? Many thanks!

onnimonni commented 7 years ago

Np! I'll report this to nginx.

onnimonni commented 7 years ago

This can be tracked over here: https://trac.nginx.org/nginx/ticket/1120#ticket

agentzh commented 7 years ago

I am afraid you are reporting to the wrong place. ngx_redis is not a standard nginx module but a 3rd party module. You should contact the author of that module instead of the official nginx team.

onnimonni commented 7 years ago

You are right and I contacted the ngx_http_redis module author: osa@FreeBSD.ORG.ru.

I'm afraid that he's not actively developing this module anymore.

agentzh commented 7 years ago

@onnimonni Hmm, seems like it's the time for OpenResty to fork it :)

onnimonni commented 7 years ago

That would be excellent :)! Can you update this thread when that happens? Then I can again use the redis module bundled in openresty package.

ksbomj commented 7 years ago

Hi guys! Is there any stable version that support redis auth?

agentzh commented 7 years ago

@ksbomj It's been supported in stable releases of ngx_redis2 for 5+ years already. I wonder why you are asking.

agentzh commented 7 years ago

@ksbomj OK, you are referring to ngx_redis. Well, you'll have to ask the author of ngx_redis instead. The ngx_srcache module is an old tech developed 5+ years ago. We're planning a much much better cache software based on OpenResty that can even serve CDN scale networks very well.

onnimonni commented 7 years ago

@agentzh the author if ngx_redis didn't move forward with this after I notified him about the issue last year.

The problem is that the version I mentioned above: https://github.com/Yongke/ngx_http_redis-0.3.7 is based on a bit older version. (Newest one is 0.3.8).

Would you accept a new forked version of ngx_redis module as part of openresty suite if I use Yongke's patches and apply them for the original 0.3.8?

I would want to include it in here and I can ask nginx if they would want to use and link to that version instead of the current one.

agentzh commented 7 years ago

@onnimonni OK, maybe we can create an openresty/redis-nginx-module repository as the fork of the original module.

wyvern8 commented 6 years ago

Hi @agentzh , i'm looking at options for caching of upstream responses, and was considering this module. I saw your comment :

_"The ngxsrcache module is an old tech developed 5+ years ago. We're planning a much much better cache software based on OpenResty that can even serve CDN scale networks very well."

Got any more info on this? My project is already built on OpenResty (which is great btw), and im just using the "X-Accel-Expires" header and caching in a second downstream OpenResty nginx at the moment. https://github.com/wyvern8/akamai-nginx

agentzh commented 6 years ago

@wyvern8 That would become a commercial cache software provided by OpenResty Inc:

https://openresty.com/