could not build the variables_hash on 1.4.1

[dctrwatson]

After upgrading from 1.2.9 to 1.4.1 receive this on startup:

[emerg] could not build the variables_hash, you should increase either variables_hash_max_size: 512 or variables_hash_bucket_size: 64

Setting variables_hash_max_size: 1024 fixes the issue, but this wasn't required for 1.2.9

Using tag v0.21

[agentzh]

In both nginx 1.2.9 and nginx 1.4.1, the default value of variables_hash_max_size is 512.

Will you provide a minimal nginx configuration that can reproduce this emerg message with nginx 1.4.1?

Thanks!

[agentzh]

Okay, checking out your nginx.conf file here: https://gist.github.com/dctrwatson/6075317

I cannot reproduce the emerg error message here with nginx 1.4.1 + ngx_srcache 0.21 + your nginx.conf. I build nginx this way:

./configure --prefix=/opt/nginx --with-http_realip_module \
        --with-http_stub_status_module \
        --add-module=$HOME/git/srcache-nginx-module

My hunch is that it could be that nginx 1.4.1 adds a lot more variables and you were also enabling quite a few modules which happens to exceed the default hash table limit. The limit was not hit with nginx 1.2.9 on your side just because nginx 1.2.9 had fewer built-in variables.

So increasing variables_hash_max_size in your nginx.conf is the right thing to do.

Regards, -agentzh

[dctrwatson]

Sorry, I guess I didn't debug this very thoroughly.

I went through a bunch of combinations of the 3rd party modules listed here: https://gist.github.com/dctrwatson/6075317#file-nginx-1-4-1-v

Though I had always kept:

--add-module=/home/vagrant/nginx/debian/modules/nginx-development-kit --add-module=/home/vagrant/nginx/debian/modules/set-misc-nginx-module --add-module=/home/vagrant/nginx/debian/modules/headers-more-nginx-module --add-module=/home/vagrant/nginx/debian/modules/nginx-echo

I just ran some more permutations and narrowed it down to this combination (in either order):

--add-module=/home/vagrant/nginx/debian/modules/nginx-echo --add-module=/home/vagrant/nginx/debian/modules/srcache-nginx-module

[agentzh]

Still cannot reproduce the error on my side with nginx 1.4.1 + ngx_echo v0.45 + ngx_srcache v0.21:

$ /opt/nginx/sbin/nginx -V
nginx version: nginx/1.4.1
built by gcc 4.7.2 20120921 (Red Hat 4.7.2-2) (GCC) 
configure arguments: --prefix=/opt/nginx --with-http_realip_module --prefix=/opt/nginx --with-http_stub_status_module --add-module=/home/agentzh/git/echo-nginx-module --add-module=/home/agentzh/git/srcache-nginx-module

[agentzh]

Okay, I can finally reproduce that [emerg] message on my side by also enabling all those extra nginx std modules:

--with-http_addition_module \
--with-http_geoip_module \
--with-http_gzip_static_module \
--with-http_image_filter_module \
--with-http_realip_module \
--with-http_spdy_module \
--with-http_stub_status_module \
--with-http_ssl_module \
--with-http_sub_module \
--with-http_xslt_module \

And analyzing the backtrace of the position that emits the [emerg] message, I can see that it's indeed caused by just too many registered built-in nginx variables:

(gdb) bt
#0  ngx_hash_init (hinit=hinit@entry=0x7fffffffdbf0, names=0x707330, nelts=124) at src/core/ngx_hash.c:315
#1  0x0000000000448441 in ngx_http_variables_init_vars (cf=cf@entry=0x7fffffffe1a0) at src/http/ngx_http_variables.c:2532
#2  0x000000000042fee0 in ngx_http_block (cf=0x7fffffffe1a0, cmd=<optimized out>, conf=<optimized out>)
  at src/http/ngx_http.c:316
#3  0x000000000041812f in ngx_conf_handler (last=1, cf=0x7fffffffe1a0) at src/core/ngx_conf_file.c:387
#4  ngx_conf_parse (cf=cf@entry=0x7fffffffe1a0, filename=filename@entry=0x6f4020) at src/core/ngx_conf_file.c:243
#5  0x0000000000415b86 in ngx_init_cycle (old_cycle=old_cycle@entry=0x7fffffffe240) at src/core/ngx_cycle.c:268
#6  0x000000000040833c in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:333

To conclude, this has nothing to do with ngx_srcache nor ngx_echo. You just have a lot of nginx built-in variables registered by both the nginx std modules and 3rd-party modules. And there's no bugs here that I can see.

Maybe the nginx core can increase the default value of variables_hash_max_size directive. 512 is just not really big enough for the rich ecosystem nowadays :)

[dctrwatson]

It just seemed weird the error occurs with 2 less 3rd party modules than we had in 1.2.9 and ngx_echo and ngx_srcache are mutually exclusive in 1.4.1 (I can run with no issues, as long as only 1 of those 2 modules is installed.) Also, that openresty 1.4.1.1 didn't run into this issue with so many more modules installed.

Thanks for your help looking into this!

I'll just leave variables_hash_max_size: 1024 in the configuration since like you said, this isn't a bug with ngx_srcache or ngx_echo.

[agentzh]

Yeah, when you add more modules you usually add more built-in nginx variables (because modules can register its own built-in nginx variables). So it's natural to hit the variables_hash_max_size limit when you add more and more modules anyway.

Because there's no bugs in ngx_srcache. I'm closing this.