failed to register channel handler while initializing push module worker on Start nginx

slact / nchan

Fast, horizontally scalable, multiprocess pub/sub queuing server and proxy for HTTP, long-polling, Websockets and EventSource (SSE), powered by Nginx.

https://nchan.io/

Other

3.01k stars 292 forks source link

failed to register channel handler while initializing push module worker on Start nginx #83

Closed blacknot closed 10 years ago

blacknot commented 11 years ago

nginx -V nginx version: nginx/1.3.7 TLS SNI support enabled configure arguments: --prefix=/usr --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --with-cc-opt=-I/usr/include --with-ld-opt=-L/usr/lib --http-log-path=/var/log/nginx/access_log --http-client-body-temp-path=/var/tmp/nginx/client --http-proxy-temp-path=/var/tmp/nginx/proxy --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi --http-scgi-temp-path=/var/tmp/nginx/scgi --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi --with-ipv6 --with-pcre --without-http_browser_module --without-http_charset_module --without-http_empty_gif_module --without-http_geo_module --without-http_limit_req_module --without-http_limit_conn_module --without-http_map_module --without-http_referer_module --without-http_scgi_module --without-http_split_clients_module --without-http_upstream_ip_hash_module --without-http_userid_module --without-http_uwsgi_module --with-http_flv_module --with-http_gzip_static_module --with-http_stub_status_module --with-http_realip_module --add-module=/var/tmp/portage/www-servers/nginx-1.3.7/work/agentzh-headers-more-nginx-module-b7c8cfc --add-module=/var/tmp/portage/www-servers/nginx-1.3.7/work/nginx_http_push_module-0.692 --with-http_ssl_module --without-mail_imap_module --without-mail_pop3_module --without-mail_smtp_module --user=nginx --group=nginx

tail -20 error_log 2012/11/23 00:04:53 [info] 12041#0: Using 32768KiB of shared memory for push module in /etc/nginx/nginx.conf:66 2012/11/23 00:04:53 [notice] 12044#0: using the "epoll" event method 2012/11/23 00:04:53 [notice] 12044#0: nginx/1.3.7 2012/11/23 00:04:53 [notice] 12044#0: OS: Linux 2.6.36-gentoo-r6 2012/11/23 00:04:53 [notice] 12044#0: getrlimit(RLIMIT_NOFILE): 1024:1024 2012/11/23 00:04:53 [notice] 12045#0: start worker processes 2012/11/23 00:04:53 [notice] 12045#0: start worker process 12046 2012/11/23 00:04:53 [notice] 12045#0: start worker process 12047 2012/11/23 00:04:53 [notice] 12045#0: start cache manager process 12048 2012/11/23 00:04:53 [notice] 12045#0: start cache loader process 12050 2012/11/23 00:04:53 [alert] 12048#0: epoll_ctl(1, 0) failed (1: Operation not permitted) 2012/11/23 00:04:53 [alert] 12048#0: failed to register channel handler while initializing push module worker (1: Operation not permitted) 2012/11/23 00:04:53 [notice] 12045#0: signal 17 (SIGCHLD) received 2012/11/23 00:04:53 [notice] 12045#0: cache manager process 12048 exited with code 2 2012/11/23 00:04:53 [alert] 12045#0: cache manager process 12048 exited with fatal code 2 and cannot be respawned 2012/11/23 00:04:53 [alert] 12050#0: epoll_ctl(1, 0) failed (1: Operation not permitted) 2012/11/23 00:04:53 [alert] 12050#0: failed to register channel handler while initializing push module worker (1: Operation not permitted) 2012/11/23 00:04:53 [notice] 12045#0: signal 17 (SIGCHLD) received 2012/11/23 00:04:53 [notice] 12045#0: cache loader process 12050 exited with code 2 2012/11/23 00:04:53 [notice] 12045#0: signal 29 (SIGIO) received

slact commented 11 years ago

does this happen on an older nginx version (stable at 1.2.5 or legacy at 1.0) as well?

ylavoie commented 11 years ago

Is does here: nginx -V nginx version: nginx/1.2.4 TLS SNI support enabled configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.2.4/debian/modules/chunkin-nginx-module --add-module=/build/buildd/nginx-1.2.4/debian/modules/headers-more-nginx-module --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-development-kit --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-http-push --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-lua --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-upload-module --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-upload-progress --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.2.4/debian/modules/nginx-dav-ext-module

ylavoie commented 11 years ago

2012/12/08 01:52:14 [info] 21531#0: Using 32768KiB of shared memory for push module in /etc/nginx/nginx.conf:181 2012/12/08 01:52:14 [alert] 21536#0: epoll_ctl(1, 0) failed (1: Operation not permitted) 2012/12/08 01:52:14 [alert] 21536#0: failed to register channel handler while initializing push module worker (1: Operation not permitted) 2012/12/08 01:52:14 [alert] 21535#0: epoll_ctl(1, 0) failed (1: Operation not permitted) 2012/12/08 01:52:14 [alert] 21535#0: failed to register channel handler while initializing push module worker (1: Operation not permitted) 2012/12/08 01:52:14 [alert] 21533#0: cache manager process 21535 exited with fatal code 2 and cannot be respawned

dimon00 commented 11 years ago

I'm having the same error. My nginx version: nginx -V nginx version: nginx/1.1.19 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-debug --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.1.19/debian/modules/chunkin-nginx-module --add-module=/build/buildd/nginx-1.1.19/debian/modules/headers-more-nginx-module --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-development-kit --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-http-push --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-lua --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-upload-module --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-upload-progress --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-dav-ext-module

mannionc commented 11 years ago

I am also having this error.

Using nginx-extras from PPA /stable branch on Ubuntu Server 12.04LTS.

$ nginx -V nginx version: nginx/1.4.1 TLS SNI support enabled configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_spdy_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.4.1/debian/modules/headers-more-nginx-module --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-cache-purge --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-dav-ext-module --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-development-kit --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.4.1/debian/modules/ngx-fancyindex --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-http-push --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-lua --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-upload-progress --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.4.1/debian/modules/ngx_http_substitutions_filter_module

Following instructions from perusio/drupal-with-nginx including those for using php5-fpm.

Output from error_log 2013/07/13 17:53:20 [alert] 3454#0: epoll_ctl(1, 0) failed (1: Operation not permitted) 2013/07/13 17:53:20 [alert] 3454#0: failed to register channel handler while initializing push module worker (1: Operation not permitted) 2013/07/13 17:53:20 [alert] 3446#0: cache manager process 3453 exited with fatal code 2 and cannot be respawned

Please help! Is there a way to reconfigure the package without building from source? dpkg-reconfigure does not seem to do the trick.

ylavoie commented 11 years ago

Modules are static and must be compiled in. Truly dynamic modules would be a huge improvement.

Yves

Envoyé de mon iPhone

Le 2013-07-13 à 16:43, mannionc notifications@github.com a écrit :

I am also having this error.

Using nginx-extras from PPA /stable branch on Ubuntu Server 12.04LTS. $ nginx -V nginx version: nginx/1.4.1 TLS SNI support enabled configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_spdy_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.4.1/d ebian/modules/headers-more-nginx-module --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-cache-purge --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-dav-ext-module --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-development-kit --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.4.1/debian/modules/ngx-fancyindex --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-http-push --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-lua --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-upload-progress --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.4.1/debian/modules/ngx_http_substitutions_filter_module

Following instructions from perusio/drupal-with-nginx including those for using php5-fpm. Output from error_log 2013/07/13 17:53:20 [alert] 3454#0: epoll_ctl(1, 0) failed (1: Operation not permitted) 2013/07/13 17:53:20 [alert] 3454#0: failed to register channel handler while initializing push module worker (1: Operation not permitted) 2013/07/13 17:53:20 [alert] 3446#0: cache manager process 3453 exited with fatal code 2 and cannot be respawned

Please help! Is there a way to reconfigure the package without building from source? dpkg-reconfigure does not seem to do the trick.

— Reply to this email directly or view it on GitHub.

slact commented 11 years ago

show me your nginx configs and the command/script you use to start nginx. I suspect this has to do with permissions of nginx master's user, or something about the number of workers and the max. number of open sockets per worker...

cwbeck commented 10 years ago

Seems like I have a similar issue. Was this ever resolved?

/var/log/nginx# cat error.log 
2013/12/04 17:21:58 [info] 20757#0: Using 32768KiB of shared memory for push module in /etc/nginx/nginx.conf:120
2013/12/04 17:21:58 [alert] 20764#0: epoll_ctl(1, 0) failed (1: Operation not permitted)
2013/12/04 17:21:58 [alert] 20764#0: failed to register channel handler while initializing push module worker (1: Operation not permitted)
2013/12/04 17:21:58 [alert] 20759#0: cache manager process 20764 exited with fatal code 2 and cannot be respawned

slact commented 10 years ago

As before, show me your nginx configs and the command/script you use to start nginx. Also the output of ulimit -a as run from the nginx user and from root. I suspect this has to do with permissions of nginx master's user, or something about the number of workers and the max. number of open sockets per worker...

sanmai commented 10 years ago

Here's a debug log with the problem:

2014/03/10 17:25:13 [debug] 2182#0: bind() [::]:80 #6 
2014/03/10 17:25:13 [debug] 2182#0: bind() [::]:443 #7 
2014/03/10 17:25:13 [notice] 2182#0: using the "epoll" event method
2014/03/10 17:25:13 [debug] 2182#0: counter: 00007F4468252080, 1
2014/03/10 17:25:13 [debug] 2182#0: posix_memalign: 0000000001471DD0:16384 @16
2014/03/10 17:25:13 [notice] 2182#0: nginx/1.4.5
2014/03/10 17:25:13 [notice] 2182#0: OS: Linux 3.12-1-amd64
2014/03/10 17:25:13 [notice] 2182#0: getrlimit(RLIMIT_NOFILE): 1024:4096
2014/03/10 17:25:13 [debug] 2183#0: write: 11, 00007FFF9C0FFE00, 5, 0
2014/03/10 17:25:13 [debug] 2183#0: setproctitle: "nginx: master process /usr/sbin/nginx"
2014/03/10 17:25:13 [notice] 2183#0: start worker processes
2014/03/10 17:25:13 [debug] 2183#0: channel 3:11
2014/03/10 17:25:13 [notice] 2183#0: start worker process 2184
2014/03/10 17:25:13 [debug] 2183#0: channel 12:13
2014/03/10 17:25:13 [notice] 2183#0: start cache manager process 2185
2014/03/10 17:25:13 [debug] 2183#0: pass channel s:1 pid:2185 fd:12 to s:0 pid:2184 fd:3
2014/03/10 17:25:13 [debug] 2183#0: channel 14:15
2014/03/10 17:25:13 [debug] 2185#0: close listening [::]:80 #6 
2014/03/10 17:25:13 [debug] 2185#0: close listening [::]:443 #7 
2014/03/10 17:25:13 [debug] 2184#0: malloc: 000000000135F190:6144
2014/03/10 17:25:13 [debug] 2184#0: malloc: 00007F44680EE010:196608
2014/03/10 17:25:13 [debug] 2184#0: malloc: 0000000001475DE0:106496
2014/03/10 17:25:13 [debug] 2184#0: malloc: 000000000148FDF0:106496
2014/03/10 17:25:13 [notice] 2183#0: start cache loader process 2186
2014/03/10 17:25:13 [debug] 2183#0: pass channel s:2 pid:2186 fd:14 to s:0 pid:2184 fd:3
2014/03/10 17:25:13 [debug] 2183#0: pass channel s:2 pid:2186 fd:14 to s:1 pid:2185 fd:12
2014/03/10 17:25:13 [debug] 2183#0: sigsuspend
2014/03/10 17:25:13 [debug] 2186#0: close listening [::]:80 #6 
2014/03/10 17:25:13 [debug] 2186#0: close listening [::]:443 #7 
2014/03/10 17:25:13 [debug] 2184#0: epoll add event: fd:6 op:1 ev:00000001
2014/03/10 17:25:13 [debug] 2185#0: malloc: 000000000135F190:6144
2014/03/10 17:25:13 [debug] 2184#0: epoll add event: fd:7 op:1 ev:00000001
2014/03/10 17:25:13 [debug] 2185#0: malloc: 0000000001475DE0:98304
2014/03/10 17:25:13 [debug] 2184#0: epoll add event: fd:8 op:1 ev:00000001
2014/03/10 17:25:13 [debug] 2185#0: malloc: 000000000148DDF0:53248
2014/03/10 17:25:13 [debug] 2185#0: malloc: 000000000149AE00:53248
2014/03/10 17:25:13 [debug] 2184#0: shmtx lock
2014/03/10 17:25:13 [debug] 2184#0: slab alloc: 56 slot: 3
2014/03/10 17:25:13 [debug] 2184#0: slab alloc: 00007F4468121080
2014/03/10 17:25:13 [debug] 2184#0: shmtx unlock
2014/03/10 17:25:13 [debug] 2184#0: epoll add event: fd:10 op:1 ev:00000001
2014/03/10 17:25:13 [debug] 2184#0: epoll add event: fd:11 op:1 ev:00000001
2014/03/10 17:25:13 [debug] 2184#0: setproctitle: "nginx: worker process"
2014/03/10 17:25:13 [debug] 2184#0: worker cycle
2014/03/10 17:25:13 [debug] 2184#0: epoll timer: -1
2014/03/10 17:25:13 [debug] 2186#0: malloc: 000000000135F190:6144
2014/03/10 17:25:13 [debug] 2184#0: epoll: fd:11 ev:0001 d:00007F44680EE310
2014/03/10 17:25:13 [debug] 2184#0: channel handler
2014/03/10 17:25:13 [debug] 2185#0: shmtx lock
2014/03/10 17:25:13 [debug] 2185#0: shmtx unlock
2014/03/10 17:25:13 [debug] 2186#0: malloc: 0000000001475DE0:98304
2014/03/10 17:25:13 [debug] 2184#0: channel: 32
2014/03/10 17:25:13 [debug] 2184#0: channel command: 1
2014/03/10 17:25:13 [debug] 2186#0: malloc: 000000000148DDF0:53248
2014/03/10 17:25:13 [debug] 2185#0: epoll add event: fd:0 op:1 ev:00000001
2014/03/10 17:25:13 [debug] 2184#0: get channel s:1 pid:2185 fd:3
2014/03/10 17:25:13 [debug] 2184#0: channel: 32
2014/03/10 17:25:13 [debug] 2184#0: channel command: 1
2014/03/10 17:25:13 [debug] 2184#0: get channel s:2 pid:2186 fd:13
2014/03/10 17:25:13 [alert] 2185#0: epoll_ctl(1, 0) failed (1: Operation not permitted)
2014/03/10 17:25:13 [debug] 2184#0: channel: -2
2014/03/10 17:25:13 [debug] 2186#0: malloc: 000000000149AE00:53248
2014/03/10 17:25:13 [debug] 2184#0: timer delta: 12
2014/03/10 17:25:13 [alert] 2185#0: failed to register channel handler while initializing push module worker (1: Operation not permitted)
2014/03/10 17:25:13 [debug] 2184#0: posted events 0000000000000000
2014/03/10 17:25:13 [debug] 2184#0: worker cycle
2014/03/10 17:25:13 [debug] 2184#0: epoll timer: -1
2014/03/10 17:25:13 [debug] 2186#0: shmtx lock
2014/03/10 17:25:13 [debug] 2186#0: shmtx unlock
2014/03/10 17:25:13 [debug] 2186#0: epoll add event: fd:0 op:1 ev:00000001
2014/03/10 17:25:13 [alert] 2186#0: epoll_ctl(1, 0) failed (1: Operation not permitted)
2014/03/10 17:25:13 [alert] 2186#0: failed to register channel handler while initializing push module worker (1: Operation not permitted)
2014/03/10 17:25:13 [notice] 2183#0: signal 17 (SIGCHLD) received
2014/03/10 17:25:13 [notice] 2183#0: cache manager process 2185 exited with code 2
2014/03/10 17:25:13 [alert] 2183#0: cache manager process 2185 exited with fatal code 2 and cannot be respawned
2014/03/10 17:25:13 [debug] 2183#0: shmtx forced unlock
2014/03/10 17:25:13 [debug] 2183#0: shmtx forced unlock
2014/03/10 17:25:13 [debug] 2183#0: shmtx forced unlock
2014/03/10 17:25:13 [debug] 2183#0: shmtx forced unlock
2014/03/10 17:25:13 [notice] 2183#0: cache loader process 2186 exited with code 2
2014/03/10 17:25:13 [debug] 2183#0: shmtx forced unlock
2014/03/10 17:25:13 [debug] 2183#0: shmtx forced unlock
2014/03/10 17:25:13 [debug] 2183#0: shmtx forced unlock
2014/03/10 17:25:13 [debug] 2183#0: shmtx forced unlock
2014/03/10 17:25:13 [debug] 2183#0: wake up, sigio 0
2014/03/10 17:25:13 [debug] 2183#0: reap children

And so forth.

Relevant bug reports: https://bugs.launchpad.net/nginx/+bug/1216817 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=732251

slact commented 10 years ago

As I described above, I need two more pieces of information to debug this: the nginx config, and the output of ulimit -a. More precisely:

I need to know the number of workers nginx is running with, and the number of open file descriptors allowed by the nginx config. I also need the results of ulimit -nS and ulimit -nH.

I suspect the problem has to do with unexpectedly running out of file descriptors, or it may be that the extra process spawned by the cache messenger is messing up the IPC initialization code in the push module. Either way, I've been unable to reproduce this so far. Having that extra info would really help me fix this.

ylavoie commented 10 years ago

I have the same problem. I get no error when Nginx is started from the console but it fails when run from Ubuntu init files.

Ulimit gives: root@rie:~# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 7792 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 7792 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited

Nginx is running with 2 workers and allows 512 connections.

I suspect that the problem lies within differences between interactive and detached processes, stderr not available, for example.

Yves

Envoyé de mon iPhone

Le 2014-03-10 à 05:39, "Leo P." notifications@github.com a écrit :

As I described above, I need two more pieces of information to debug this: the nginx config, and the output of ulimit -a. More precisely:

I need to know the number of workers nginx is running with, and the number of open file descriptors allowed by the nginx config. I also need the results of ulimit -nS and ulimit -nH.

I suspect the problem has to do with unexpectedly running out of file descriptors, or it may be that the extra process spawned by the cache messenger is messing up the IPC initialization code in the push module. Either way, I've been unable to reproduce this so far. Having that extra info would really help me fix this.

— Reply to this email directly or view it on GitHub.

slact commented 10 years ago

Thanks, this is helpful.

sanmai commented 10 years ago

This is the shortest configuration file that gives me the error:

worker_processes  1;

error_log  /tmp/nginx-error.log error;
pid        /tmp/nginx-debug.pid;

events {
    worker_connections 8;
}

http {
    proxy_cache_path /tmp levels=1:2 keys_zone=cache:1m;

    server {
        listen [::]:8000;
        location / {
            proxy_cache cache;
        }
    }
}

Testing it from under a non-privileged user with:

/usr/sbin/nginx -c `pwd`/nginx.conf; sleep 1; cat /tmp/nginx-error.log; rm /tmp/nginx-error.log; killall nginx

Output:

2014/03/11 11:13:46 [alert] 11652#0: epoll_ctl(1, 0) failed (1: Operation not permitted)
2014/03/11 11:13:46 [alert] 11652#0: failed to register channel handler while initializing push module worker (1: Operation not permitted)
2014/03/11 11:13:46 [alert] 11654#0: epoll_ctl(1, 0) failed (1: Operation not permitted)
2014/03/11 11:13:46 [alert] 11654#0: failed to register channel handler while initializing push module worker (1: Operation not permitted)
2014/03/11 11:13:46 [alert] 11650#0: cache manager process 11652 exited with fatal code 2 and cannot be respawned

ulimit -a

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 126687
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 32000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 126687
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

/usr/sbin/nginx -V

nginx version: nginx/1.4.5
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_spdy_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-mail --with-mail_ssl_module --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/headers-more-nginx-module --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-auth-pam --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-cache-purge --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-dav-ext-module --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-development-kit --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-echo --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/ngx-fancyindex --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-http-push --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-lua --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-upload-progress --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/nginx-upstream-fair --add-module=/tmp/buildd/nginx-1.4.5/debian/modules/ngx_http_substitutions_filter_module

sanmai commented 10 years ago

This one also gives the error:

worker_processes  1;

error_log  /tmp/nginx-error.log error;
pid        /tmp/nginx-debug.pid;

events {
    worker_connections 8;
}

http {
    proxy_cache_path /tmp levels=1:2 keys_zone=cache:1m;
}

Error output is same as above.

If I leave out proxy_cache_path the error goes away, same as the cache.

slact commented 10 years ago

Ideal bug report. Thanks!

slact commented 10 years ago

I think this is fixed in master (836e8319c93681386fb00e6bd34d9e37612f3334). Could someone please double-check against the master branch for my own sanity?

sanmai commented 10 years ago

At the very least I can tell that the error is gone. Though I didn't check everything.

slact commented 10 years ago

Ok, thanks. It passes my tests fine. Fixed in v0.711

s-a-s-k-i-a commented 8 years ago

seeing this error with current nginx mainline and fastcgi cache enabled. error.log

Using 32768KiB of shared memory for push module in /etc/nginx/nginx.conf:114

my config:

root@server:~# nginx -V
nginx version: nginx/1.9.9
built with OpenSSL 1.0.2e 3 Dec 2015
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_random_index_module --with-http_secure_link_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-http_slice_module --with-mail --with-mail_ssl_module --with-stream --with-stream_ssl_module --with-threads --add-module=/usr/src/packages/BUILD/debian/modules/headers-more-nginx-module --add-module=/usr/src/packages/BUILD/debian/modules/nginx-auth-pam --add-module=/usr/src/packages/BUILD/debian/modules/nginx-cache-purge --add-module=/usr/src/packages/BUILD/debian/modules/nginx-dav-ext-module --add-module=/usr/src/packages/BUILD/debian/modules/nginx-echo --add-module=/usr/src/packages/BUILD/debian/modules/ngx-fancyindex --add-module=/usr/src/packages/BUILD/debian/modules/nginx-http-push --add-module=/usr/src/packages/BUILD/debian/modules/nginx-lua --add-module=/usr/src/packages/BUILD/debian/modules/nginx-upload-progress --add-module=/usr/src/packages/BUILD/debian/modules/nginx-upstream-fair --add-module=/usr/src/packages/BUILD/debian/modules/ngx_http_substitutions_filter_module --add-module=/usr/src/packages/BUILD/debian/modules/memc-nginx-module --add-module=/usr/src/packages/BUILD/debian/modules/srcache-nginx-module --add-module=/usr/src/packages/BUILD/debian/modules/HttpRedisModule --add-module=/usr/src/packages/BUILD/debian/modules/redis2-nginx-module --add-module=/usr/src/packages/BUILD/debian/modules/nginx-development-kit --add-module=/usr/src/packages/BUILD/debian/modules/set-misc-nginx-module --add-module=/usr/src/packages/BUILD/debian/modules/ngx_pagespeed

This is my nginx.conf from line 0-114:

user www-data;
worker_processes 8;
worker_rlimit_nofile 100000;
pid /run/nginx.pid;

events {
    worker_connections 4096;
    multi_accept on;
}

http {
    ##
    # Special settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 30;
    types_hash_max_size 2048;

    server_tokens off;
    reset_timedout_connection on;
    add_header X-Powered-By "Special Settings 1.0";
    add_header rt-Fastcgi-Cache $upstream_cache_status;

    # Limit Request
    limit_req_status 403;
    limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;

    # Proxy Settings
    # set_real_ip_from  proxy-server-ip;
    # real_ip_header    X-Forwarded-For;

    fastcgi_read_timeout 300;
    client_max_body_size 100m;

    ##
    # SSL Settings
    ##

    ssl_session_cache shared:SSL:20m;
    ssl_session_timeout 10m;
    ssl_prefer_server_ciphers on;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!CAMELLIA;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_dhparam /etc/ssl/certs/dhparams.pem;

    ##
    # Basic Settings
    ##
    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    # Log format Settings
    log_format rt_cache '$remote_addr $upstream_response_time $upstream_cache_status [$time_local] '
    '$http_host "$request" $status $body_bytes_sent '
    '"$http_referer" "$http_user_agent"';

    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "msie6";

    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types
        application/atom+xml
        application/x-javascript
        application/javascript
        application/json
        application/rss+xml
        application/vnd.ms-fontobject
        application/x-font-opentype
        application/x-font-truetype
        application/x-font-ttf
        application/x-web-app-manifest+json
        application/xhtml+xml
        application/xml
        font/opentype
        font/otf
        font/eot
        image/svg+xml
        image/vnd.microsoft.icon
        image/x-icon
        text/css
        text/plain
        text/x-component
        text/xml
       # text/html
        text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Do you need anymore info? user www-data is user id 33 group id 33

slact commented 8 years ago

It looks like you're using Debian's nginx-extras package, which bundles the Nginx HTTP Push module, an old version of Nchan. The logged message is not an error.

vikasnetbeam commented 7 years ago

i am getting this error: 2016/12/24 00:03:04 [notice] 12849#0: using inherited sockets from "6;7;8;" 2016/12/24 00:03:07 [info] 12876#0: Using 32768KiB of shared memory for push module in /etc/nginx/nginx.conf:59

user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
    worker_connections 768;
    # multi_accept on;
}
http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    # server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # set client body size to 2M #
    client_max_body_size 50M;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "msie6";

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
 }`

slact commented 7 years ago

@vikasnetbeam : What version of nginx & nchan are you using? These are notices from an old version of nchan / push module. They are also not errors.