search

wandenberg / nginx-push-stream-module

A pure stream http push technology for your Nginx setup. Comet made easy and really scalable.
Other
2.21k stars 295 forks source link

Diffrent value of Writing and subscribers #143

Closed kamil-michalak closed 9 years ago

kamil-michalak commented 10 years ago

After few days of working nginx (1.6.0) with push stream module (latest dev brach from git) i have some diffrence between status page from nginx status module http://wiki.nginx.org/HttpStubStatusModule and push stream stat page.

At start value of Writing and subscribers was equal, but after few days value of subscribers grow up. example Reading: 11 Writing: 19258 Waiting: 1984

subscribers 7982 /subscribers

wandenberg commented 10 years ago

Can you test if this commit 3bd35e6470fc06cb5db4e54f7229e9ecee968066 solve your problem?

kamil-michalak commented 10 years ago

I will check

kamil-michalak commented 10 years ago

Problem still exists push: subscribers: 7160 uptime: 82771

nginx stats Active connections: 12024 server accepts handled requests 9484887 9484887 76187310 Reading: 0 Writing: 9089 Waiting: 2935

wandenberg commented 10 years ago

Can you send me your configuration? Are you using other 3rd party modules? Are you sure that you used the commit I suggested? Another person had the same problem and that commit solved it.

If possible describe how are using the module, streaming modes in use, if you are using the function of delete channels, if use the channels stats frequently , ... any information can help check if the problem is inside the module

wandenberg commented 10 years ago

Hi @kilgor,

do you still having this problem?

kamil-michalak commented 9 years ago

Yes Problem still exists

Active connections: 123115 server accepts handled requests 445507337 445507337 3583595779 Reading: 1 Writing: 117229 Waiting: 5885

time>2014-07-17T14:06:30 channels>0 wildcard_channels>1 published_messages>63035 stored_messages>5 messages_in_trash>0 channels_in_trash>0 subscribers>13575 uptime>3644660

wandenberg commented 9 years ago

Hi @kilgor

can you answer my questions on previous message (8/Jun)? I need more info to try to help you, since only you are reporting the issue.

kamil-michalak commented 9 years ago

Currently i make debian package from dotdeb source and i use git version of module.

apt-get update apt-get source nginx cd nginx-* rm -r debian/modules/nginx-push-stream-module cd debian/modules

git clone -b 0.4.1-dev https://github.com/wandenberg/nginx-push-stream-module.git

git clone https://github.com/wandenberg/nginx-push-stream-module.git cd ../../ dpkg-buildpackage

dotdeb source: .... deb http://packages.dotdeb.org wheezy all deb-src http://packages.dotdeb.org wheezy all

And i use debian-extra package

nginx version: nginx/1.6.0 built by gcc 4.7.2 (Debian 4.7.2-5) TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt=-Wl,-z,relro --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-file-aio --with-http_spdy_module --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_gunzip_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_secure_link_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/headers-more-nginx-module --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/naxsi/naxsi_src --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-auth-ldap --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-auth-pam --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-cache-purge --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-dav-ext-module --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-development-kit --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-echo --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/ngx-fancyindex --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-push-stream-module --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-lua --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-upload-progress --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-upstream-fair --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-syslog --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/ngx_http_pinba_module --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/ngx_http_substitutions_filter_module --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/ngx_pagespeed --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-stable/nginx-1.6.0/debian/modules/nginx-x-rid-header --with-ld-opt=-lossp-uuid

nginx.conf: push_stream_shared_memory_size 1024m; push_stream_max_channel_id_length 200; push_stream_message_ttl 5m; push_stream_ping_message_interval 25s; push_stream_subscriber_connection_ttl 15m; push_stream_longpolling_connection_ttl 70s;

push_stream_broadcast_channelprefix "bd";

push_stream_wildcard_channelprefix "bd";

push_stream_message_template "{\"id\":~id~,\"channel\":\"~channel~\",\"text\":\"~text~\",\"eventid\":\"~event-id~\"}"; push_stream_authorized_channels_only on;

site.conf location ~ /sub/(.*) {

activate subscriber mode for this location

push_stream_subscriber;
# positional channel path
#set $push_stream_channels_path              $1;
push_stream_channels_path $1;

# header to be sent when receiving new subscriber connection
push_stream_header_template                 "<html><head><meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\">\r\n<meta http-equiv=\"Cache-Control\" content=\"no-store\">\r\n<meta http-equiv=\"Cache-Control\
# message template
#push_stream_message_template                "<script>p(~id~,'~channel~','~text~','~event-id~');</script>";
push_stream_message_template                "<script>p(~id~,'~channel~','~text~');</script>";
# footer to be sent when finishing subscriber connection
push_stream_footer_template                 "</body></html>";
# content-type

<-----> push_stream_content_type "text/html; charset=utf-8";

}

location ~ /ev/(.*) {

activate subscriber mode for this location

<-----> push_stream_subscriber;

push_stream_subscriber eventsource;
# positional channel path
#set $push_stream_channels_path              $1;
push_stream_channels_path $1;

# activate event source support for this location

<-----> push_stream_eventsource_support on;

}

location ~ /lp/(.*) {

activate long-polling mode for this location

push_stream_subscriber      long-polling;
# positional channel path

<-----> set $push_stream_channels_path $1;

push_stream_channels_path $1;

}

location ~ /ws/(.*) {

activate websocket mode for this location

#push_stream_websocket;
push_stream_subscriber websocket;
# positional channel path
#set $push_stream_channels_path              $1;
push_stream_channels_path $1;

# store messages in memory
push_stream_store_messages              on;

push_stream_websocket_allow_publish     on;

}

location /t1/statsXXXXXXXXXXXXXXXXXXXXXXXXXXX { push_stream_channels_statistics; push_stream_channels_path $arg_id; }

wandenberg commented 9 years ago

Which other modules are you using in this conf? Can you send me you complete conf? (may be in private) This conf you sent seems incomplete. If possible, split your server into two instances, one only for stream, and the other to your other things. There are so many modules on your setup that any of them can be causing the problem, not necessarily mine. :)

kamil-michalak commented 9 years ago

Currently i have small number of modules in config

nginx -V nginx version: nginx/1.6.0 built by gcc 4.7.2 (Debian 4.7.2-5) TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt=-Wl,-z,relro --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-file-aio --with-http_spdy_module --with-http_gzip_static_module --without-http_browser_module --without-http_memcached_module --without-http_scgi_module --without-http_ssi_module --without-http_uwsgi_module --with-http_secure_link_module --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-custom/nginx-1.6.0/debian/modules/nginx-echo --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-custom/nginx-1.6.0/debian/modules/nginx-x-rid-header --add-module=/home/conf/build/nginx/64/nginx_1.6.0-1-custom/nginx-1.6.0/debian/modules/nginx-push-stream-module --with-ld-opt=-lossp-uuid

Need 2 days to test all

kamil-michalak commented 9 years ago

You fixed this bug on cdc568981309f2bd5bde599d874103151943cde7