slact / nchan

Fast, horizontally scalable, multiprocess pub/sub queuing server and proxy for HTTP, long-polling, Websockets and EventSource (SSE), powered by Nginx.
https://nchan.io/
Other
2.99k stars 292 forks source link

Incorrect "channels" stat when using multiplexed channels #673

Closed mkdewidar closed 4 months ago

mkdewidar commented 11 months ago

It seems that when a multiplexed channel connects, both the "memstore_multi" channel head and its child "memstore" channel heads count towards the "channels" statistic, instead of just the child channel heads.

In our use case, where a subscriber connects to either 1 or 2 multiplexed channels. We're finding that if we create a multiplexed subscription, the channels increase by 3, whereas if we create a single subscription it goes up by 1 (as expected).

NChan example config

location ~ /sub/([-,\w]+)$ {

  nchan_subscriber;
  nchan_subscriber_first_message newest;
  nchan_channel_id $1;
  nchan_channel_id_split_delimiter ",";
  nchan_redis_pass redis_cluster;
  nchan_eventsource_ping_interval 60;
  nchan_eventsource_ping_comment "ping";
  nchan_eventsource_ping_event "";
  nchan_subscribe_request /notify/sub;
  nchan_unsubscribe_request /notify/unsub;
}

location ~ "/pub/([-,\w]+)$" {

  nchan_publisher;
  nchan_channel_id $1;
  nchan_redis_pass redis_cluster;
  nchan_channel_id_split_delimiter ",";
}

location /nchan-stats {

  nchan_stub_status;
}

When we connect with /sub/foo,bar and then check the status we see:

total published messages: 1
stored messages: 0
shared memory used: 16K
shared memory limit: 262144K
channels: 3
subscribers: 1
redis pending commands: 0
redis connected servers: 3
redis unhealthy upstreams: 0
total redis commands sent: 237
total interprocess alerts received: 0
interprocess alerts in transit: 0
interprocess queued alerts: 0
total interprocess send delay: 0
total interprocess receive delay: 0
nchan version: 1.3.6

But if we then connect with /sub/far and then check the status:

total published messages: 1
stored messages: 0
shared memory used: 16K
shared memory limit: 262144K
channels: 4
subscribers: 2
redis pending commands: 3
redis connected servers: 3
redis unhealthy upstreams: 0
total redis commands sent: 242
total interprocess alerts received: 0
interprocess alerts in transit: 0
interprocess queued alerts: 0
total interprocess send delay: 0
total interprocess receive delay: 0
nchan version: 1.3.6

We expect the "channels" metric to match the total number of individual channels that we've subscribed to, so in this case 3 (foo, bar and far).

slact commented 4 months ago

That's the expected behavior. If a client subscribes to a multiplexed channel, Nchan creates the demuxed channels individually, and joins them together in the multiplexed one.

Let's say, I have a subscriber multiplexing channels foo and bar. There are two message buffers, but three places to receive their data. Thus, Nchan counts this as 3 channels. (Also, as you've seen, 3 channels are allocated).

It may be odd but that's how Nchan counts channels for statistical purposes.