shared memory zone "push_stream_module" was locked error when very high concurrent

[ideawu]

We are using nginx-push-stream-module on very hight concurrent enviroment, about 50000 channels and 60000 subscribers, then nginx/push-stream goes nearly unavailable.

Subs are reset after seconds, pubs take several miniutes to finished. Getting push_stream_channels_statistics through HTTP also takes very long time, and returns 0 subscribers, uptime 0. ps aux | grep nginx shows that nginx keeps spawn master process and very soon the spawned master process exit.

here's nginx error.log output:

2013/07/13 11:30:50 [alert] 1153#0: shared memory zone "push_stream_module" was locked by 8949
2013/07/13 11:30:56 [alert] 1153#0: worker process 8955 exited on signal 11
2013/07/13 11:30:56 [alert] 1153#0: shared memory zone "push_stream_module" was locked by 8955
2013/07/13 11:30:56 [alert] 1153#0: worker process 8956 exited on signal 11
2013/07/13 11:30:56 [alert] 1153#0: shared memory zone "push_stream_module" was locked by 8956

[wandenberg]

What nginx version are you using? What push stream version are you using? How many workers? Are you using any other third party modules? Which ones? Try to generate a core dump to be possible find out why your workers are dying. Modules are not design to work well when nginx workers die, we should check why this is happening before check any other problem. We have applications running in a heavier scenario concerning concurrency.

[ideawu]

1. nginx version: nginx/1.2.7
2. push stream version: Version 0.3.5
3. only 1 worker
4. nginx configure args:

$ usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.2.7
built by gcc 4.1.2 20080704 (Red Hat 4.1.2-46)
TLS SNI support disabled
configure arguments: --prefix=/usr/local/nginx-1.2.7 --with-http_sub_module --with-http_stub_status_module --with-http_ssl_module --with-http_gzip_static_module --add-module=exp/echo-nginx-module/ --add-module=exp/lua-nginx-module/ --add-module=exp/simpl-ngx_devel_kit/ --add-module=exp/set-misc-nginx-module/ --add-module=exp/ngx_cache_purge/ --add-module=exp/nginx-push-stream-module --with-ld-opt=-ldl

This is as much as I know.

[wandenberg]

Hi,

as I said, any other module could be causing the worker's dead. Try to generate a core dump, see on http://wiki.nginx.org/Debugging how to do. Don't forget to set debug symbols. With this will be possible check which code are causing the problem.

Consider increase the number of workers too, you probably are getting a high load on that process.

[wandenberg]

Hi, did you get any core dump about this problem?

[ideawu]

@wandenberg Hi, I simply added more servers for nginx and push-stream regardless of this problem.