search

feathersjs-ecosystem / feathers-sync

Synchronize service events between Feathers application instances
MIT License
222 stars 41 forks source link

Reported app.channel(channel).connections incorrect with multiple instances of feathers-sync #183

Open DelfsEngineering opened 1 year ago

DelfsEngineering commented 1 year ago

Steps to reproduce

When using multiple instances, you can not get a list of all connections / users. app.channel(channel).connections only contains connections on that instance and not the whole network of users. I suspect this is because that instance only stores the broadcast message for its connected users?

Expected behavior

See all connected users in app.channel(channel).connections

Actual behavior

Only users connected to the specific instance are returned

In a network of 3 servers where server 1 contains [ Charles, Eduardo ] server 2 contains [ Cindy ] server 3 contains [ ] // no connected users

querying server 2 app.channel(channel).connections would return [ Cindy ] only.

feathers-sync version: 2.1.0

Perhaps there is a better method for getting all connections/ users that can be used? Note, in an auto-scaling network, it is unknown how many instances (pods) are currently active. Also, it does not look like REDIS can be queried for connections as it seems to only be used as a broadcasting bus.

daffl commented 1 year ago

Feathers tries to provide as stateless of an architecture as possible so while there are persistent channel connections, they should only be used to distribute them into channels, not to manage any other overall application state.

I'd recommend that if there is state to be shared it should be persisted to the database or another storage mechanism that's shared between all instances (e.g. Redis).

OnnoGabriel commented 9 months ago

Similar request here. I need all connections for a channel, regardless in which Feathers instance the user has joined that channel.

@daffl, will this forever be out of the scope of feathers-sync? And if yes, would the following be an approach for a custom implementation?

  1. Use app.on('connection') and app.on('disconnect') to watch for any change in the connections in each app instance.
  2. Get a list of channels and save for each channel the array of connections in that instance to a persistant storage (e.g. with [channel-id]/[instance-id] as a key).
  3. To fetch all connections in all instances, we would have to fetch all values for [channel-id]/* from the storage and return the concatenated arrays.
daffl commented 9 months ago

Connections are object references to the actual sockets on an instance so they can't be saved to a persistent storage. Storing all connected clients by their id in a key-value store or database and removing them on disconnect should already be possible with the existing channel events (https://feathersjs.com/api/channels.html#app-on-connection).

OnnoGabriel commented 9 months ago

Ok, thanks! I do not need the full object references to the actuell sockets, just some information about the connected clients. So, storing that information in Redis together with channel and app instance IDs should be fine. Will give it a try!