search

wandenberg / nginx-push-stream-module

A pure stream http push technology for your Nginx setup. Comet made easy and really scalable.
Other
2.22k stars 295 forks source link

Worker process exited on signal 11 (core dumped) #216

Closed MaximChernomorov closed 8 years ago

MaximChernomorov commented 8 years ago

Hello.

module version 0.5.1 nginx version 1.9.5

push_stream_shared_memory_size 2048M;
push_stream_message_ttl 120;
push_stream_max_messages_stored_per_channel 4000;
push_stream_max_channel_id_length 32;
push_stream_max_number_of_channels 400000;

So we keep getting core dump every 24 hours. Unfortunately, we cant use https://www.nginx.com/resources/wiki/start/topics/tutorials/debugging/ ( log for 1 hour weighs more than 1gb)

/channel-stats right before 

{"hostname": "example.com", "time": "2015-11-16T21:14:03", "channels": "4",
 "wildcard_channels": "0", "published_messages": "837", "stored_messages": "1", "messages_in_trash": "1", 
 "channels_in_trash": "2", "subscribers": "2", "uptime": "76209", "by_worker": [
{"pid": "3279", "subscribers": "0", "uptime": "3475"},
{"pid": "5312", "subscribers": "0", "uptime": "2995"},
{"pid": "6276", "subscribers": "0", "uptime": "2636"},
{"pid": "3924", "subscribers": "0", "uptime": "3237"},
{"pid": "28658", "subscribers": "0", "uptime": "5756"},
{"pid": "5637", "subscribers": "0", "uptime": "2875"},
{"pid": "5951", "subscribers": "0", "uptime": "2755"},
{"pid": "2281", "subscribers": "0", "uptime": "3836"},
{"pid": "5313", "subscribers": "2", "uptime": "2995"},
{"pid": "9502", "subscribers": "0", "uptime": "1436"},
{"pid": "4233", "subscribers": "0", "uptime": "3116"},
{"pid": "1918", "subscribers": "0", "uptime": "3956"}
]}

right after

{"hostname": "example.com", "time": "2015-11-16T21:15:01",
 "channels": "4", "wildcard_channels": "0", "published_messages": "837", "stored_messages": "1", "messages_in_trash": "0",
 "channels_in_trash": "0", "subscribers": "0", "uptime": "76267", "by_worker": [
]}

Can u give any advice?

wandenberg commented 8 years ago

Hi @biskvit

to make our job easier move to master branch code, please. There are some important changes between 0.5.1 and master code. In fact you can follow only the core-dump part of the debugging tutorial. You don't need to change the log level or complile you nginx with the --with-debug parameter. Only set the CFLAGS values and if you are using a package system, make sure that the symbols are not being stripped from the package. This way we will have a good core dump to check where and why your worker is dying out. If you have others modules on the same build may be any of them is causing the worker death, not necessarily the push stream module, but cause the module to not work well from that point.

MaximChernomorov commented 8 years ago

In fact you can follow only the core-dump part of the debugging tutorial.

we did only core dump logs. They are becoming too big too fast and making load for disks.

If you have others modules on the same build may be any of them is causing the worker death, not necessarily the push stream module, but cause the module to not work well from that point.

2015/11/17 00:15:01 [info] 14425#14425: Using 2097152KiB of shared memory for push stream module on zone: push_stream_module

we disabled push stream module and workers stopped dying

Thanks for awnser. We'll try use master branch and maybe somehow get core dump log

MaximChernomorov commented 8 years ago

Seems not a problem of module, anyway thank you for help.